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Abstract: This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control 
problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows 
that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the 
corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. 
Conditions ensuring the convexity of the cost functional are discussed, including the issue that how negative 
the control weighting matrix-valued function R{-) can be. Finiteness of the LQ problem is characterized 
by the convergence of the solutions to a family of Riccati equations. Then, a minimizing sequence, whose 
convergence is equivalent to the open-loop solvability of the problem, is constructed. Finally, an illustrative 
example is presented. 

Keywords: linear quadratic optimal control, stochastic differential equation, Riccati equation, finiteness, 
open-loop solvability, closed-loop solvability. 

AMS Mathematics Subject Classification. 49N10, 49N35, 93E20. 


1 Introduction 


Let (n, J-, F, P) be a complete filtered probability space on which a standard one-dimensional Brownian 
motion W = {lY(t);0 < t < 00 } is defined, where F = {J5}t>o is the natural filtration of W augmented 
by all the P-null sets in R [16, 30]. Consider the following controlled linear stochastic differential equation 
(SDE, for short) on a finite horizon [t,T]: 


( 1 . 1 ) 


dX{s) = [A(s)A(s) -I- B{s)u{s) + 6(s)]c?s -I- [C'(s)A(s) -|- D{s)u{s) + tT(s)]dVF(s), 
X{t) = X, 


s € [t, T], 


where A{-), D{-) are given deterministic matrix-valued functions of proper dimensions, and b{-),<7{-) 

are vector-valued F-progressively measurable processes. In the above, A(-), valued in K", is the state process, 
and u(-)j valued in R"*, is the control process. For any t G [0,T), we introduce the following Hilbert space: 


U[t,T] = lu:[t,T]xn 


u{-) is F-progressively measurable, E / |M(s)pds < 00 
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Any u(-) € W[t, T] is called an admissible control (on [t,T]). Under some mild conditions on the coefficients, 
for any initial pair {t,x) € [0,T) x R" and admissible control u{-) G U\t^T]^ (1-1) admits a unique strong 
solution X{-) = X{- ;t,x,u{-)). Next we introduce the following cost functional: 


J{t, x; u{-)) 4 E<' {GX{T),X{T)) + 2{g, X{T)) 

(q{s) S{sY\(x[s)\ 

[sis) R{s) ) [uis) ) ’ \u{s))l \\p{s)) ’ [uis) ) 




where G is a symmetric matrix, Q{-), S{-), R{-) are deterministic matrix-valued functions of proper dimen¬ 
sions with = Q{-), = -R('); 9 is allowed to be an J^^-measurable random variable and g(-),p(-) 

are allowed to be vector-valued F-progressively measurable processes. The classical stochastic LQ optimal 
control problem can be stated as follows. 


Problem (SLQ). For any given initial pair {t,x) G [0,T) x R", find a u*{-) G U[t,T], such that 


(1.3) J{t,x\u*{-)) = inf J(t,x-,u{-)) = V{t,x). 

It is well-accepted that any u*{-) G lA[t,T] satisfying (1.3) is called an optimal control of Problem (SLQ) 
for the initial pair {t^x), and the corresponding A*(-) = X{- ;t,x,u*{-)) is called an optimal state process; 
the pair (X*(-), u*{-)) is called an optimal pair. The function V{- ,■) is called the value function of Problem 
(SLQ). When b{-),a{-),g,q{-), p{-) = 0, we denote the corresponding Problem (SLQ) by Problem (SLQ)°. 
The corresponding cost functional and value function are denoted by J°(t, x; u{-)) and U°(t, x), respectively. 

When the stochastic part is absent, with 6(-), g, g(-) and p(-) being deterministic. Problem (SLQ) is 
reduced to a deterministic LQ optimal control problem, called Problem (DLQ). Hence, Problem (DLQ) 
can be regarded as a special case of Problem (SLQ). The history of Problem (DLQ) can be traced back 
to the work of Bellman-Glicksberg-Gross [5] in 1958, Kalman [15] in 1960 and Letov [18] in 1961. In the 
deterministic case, it is well known that R{s) ^ 0 is necessary for Problem (DLQ) to be finite (meaning that 
the infimum of the cost functional is finite). When the control weighting matrix R{s) in the cost function 
is uniformly positive definite, under some mild additional conditions on the other weighting coefficients, the 
problem can be solved elegantly via the Riccati equation; see [4] for a thorough study of the Riccati equation 
approach (see also [30]). Stochastic LQ problems was firstly studied by Wonham [26] in 1968, followed by 
several researchers later (see [12, 6], for examples). In those works, the assumption that R{s) > 0 was taken 
for granted. More precisely, under the following standard conditions: 


(1.4) G>0, R{s)^SI, Q{s)-S{syR{s)-^S{s)^0, a.e. se[0,T], 


for some d > 0, the corresponding Riccati equation is uniquely solvable and Problem (SLQ) admits a unique 
optimal control which has a linear state feedback representation (see [30, Chapter 6] or [10]). In 1998, Chen- 
Li-Zhou [7] found that Problem (SLQ) might still be solvable even if R{s) is not positive semi-definite. See 
also some follow-up works of Lim-Zhou [19], Chen-Zhou [10], and Chen-Yong [9], as well as the works of 
Hu-Zhou [13] and Qian-Zhou [23] on the study of solvability of indefinite Riccati equations (under certain 
technical conditions). In 2001, Ait RamUMoore-Zhou [1] introduced a generalized Riccati equation involving 
the pseudo-inverse of a matrix and an additional algebraic constraint; see also Ait Rami-Zhou-Moore [2] for 
stochastic LQ optimal control problem on [0, oo), and a follow-up work of Wu-Zhou [27]. Recently, based on 
the work of Yong [28], Huang-Li-Yong [14] studied a mean-field LQ optimal control problem on [0, oo). For 
stochastic LQ optimal control problems with random coefficients, we further refer to the works of Chen-Yong 
[8], Tang [25], and Kohlmann-Tang [17]. 
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Most recently, Sun-Yong [24] established that the existence of open-loop optimal controls is equivalent 
to the solvability of the corresponding optimality system which is a forward-backward stochastic differential 
equation (FBSDE, for short), and the existence of closed-loop optimal strategies is equivalent to the existence 
of a regular solution to the corresponding Riccati equation. From this point of view, this paper can be 
regarded as a continuation of [24], in a certain sense. Inspired by a result found in [29], we are able to cook 
up an example for which open-loop optimal controls exist, but the closed-loop optimal strategy does not 
exist. Because of this, it is necessary to distinguish open-loop and closed-loop solvabilities of Problem (SLQ). 
Next, having the equivalence between the solvability of the Riccati equation and the closed-loop solvability of 
Problem (SLQ), it is natural to seek conditions under which the Riccati equation is solvable, and the sought 
conditions are expected to be more general than (1.4) so that they could include some (although might not 
be all) cases that R{-) is allowed to be not positive semi-definite. One of our main results in this paper is 
to establish the equivalence between the strongly regular solvability of the Riccati equation (see below for a 
definition) and the uniform convexity of the cost functional. Note that uniform convexity condition is much 
weaker than (1.4), and is different from conditions imposed in [23]. 

Finiteness of Problem (SLQ) (meaning that the infimum of the cost functional is finite) is another 
important issue. The notion was introduced in [30] (see also [8, 9]). Some investigations were carried out in 
[21]. In this paper, due to the perfect structure of Problem (SLQ)°, its finiteness will be characterized by the 
convergence of the solutions to a family of Riccati equations. As a by-product, we will construct minimizing 
sequences of Problem (SLQ) in a very natural way, and the convergence of the sequences will lead to the 
open-loop solvability of Problem (SLQ). 

Among other things, we find several interesting facts which are listed below: 

Fact 1. The value function V{t,x) is not necessarily continuous in t even if Problem (SLQ) admits a 
continuous open-loop optimal control at all initial pair (t,x) G [0,T) x R”. 

Fact 2. If Problem (SLQ)° is finite at t, then it is finite at any t' > t. 

Fact 3. For Problem (SLQ) with D{-) = 0, under the assumption that R{-) is uniformly positive definite, 
without assuming the non-negativity of Q{-) and G, the finiteness and the unique closed-loop solvability of 
Problem (SLQ) are equivalent, which are also equivalent to the uniform convexity of the cost functional. 

Fact 4. The existence of a regular solution to the Riccati equation implies the open-loop solvability of 
Problem (SLQ). However, it may happen that for any initial pair (t, x) € [0, T) x R”, Problem (SLQ) admits 
a continuous open-loop optimal control, while the Riccati equation does not admit a regular solution. This 
corrects an incorrect result found in [1] (see Section 4 for details). 

The rest of the paper is organized as follows. Section 2 collects some preliminary results. In Section 3, we 
study the cost functional from a Hilbert space viewpoint and represent it as a quadratic functional of u{-). 
Section 4 is devoted to the strongly regular solvability of the Riccati equation under the uniform convexity 
of the cost functional. In Section 5, we discuss the finiteness of Problem (SLQ) as well as the convexity of 
the cost functional. In Section 6, we characterize the open-loop solvability of Problem (SLQ) by means of 
the convergence of minimizing sequences. An example is presented in section 7 to illustrate some relevant 
results obtained. 


2 Preliminaries 

We recall that R" is the n-dimensional Euclidean space, R'^^™ is the space of all (n x m) matrices, endowed 
with the inner product {M,N) >->• tr [M^N] and the norm \M\ = y'tr [M^M], C R"^" is the set of all 
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(n X n) symmetric matrices, S” C S" is the set of all [n x n) positive semi-definite matrices, and S" C 
is the set of all (n x n) positive-definite matrices. When there is no confusion, we shall use (•, ■) for inner 
products in possibly different Hilbert spaces. Also, stands for the (Moore-Penrose) pseudo-inverse of the 
matrix M ([22]), and TZ{M) stands for the range of the matrix M. Next, let T > 0 be a fixed time horizon. 
For any t G [0, T) and Euclidean space H, let 


C{[t,T];M) = |(/? : [t,T] ^ H | (p{-) is continuous |, 


L°“(t,r;H) = L:[t,T] 


|(/7(s)|^ds < oo| , 1 ^ p < oo. 


esssup |(p(s)| < oo > . 
se[t.T] I 


We denote 

H —>• HI I is J^r-measurable, < oo|, 

Lf{t,T;M) = |(p : [t,T] x fl —H | ip(-) is F-progressively measurable, E ^ |(p(s)p(is < oo| , 
T]; H)) = < (p : [t, T] x H —)• HI I (p{-) is F-adapted, continuous, E 


sup |(p(s)p 
se[t,T] 


< oo 

2 


Ap(H;L^(t,T;HI)) = < p : [t, T] x fl —>■ HI | (p(-) is F-progressively measurable,E |(p(s)|ds^ < 



For an S"-valued function F{-) on [t,T], we use the notation F{-) ^ 0 to indicate that F{-) is uniformly 
positive definite on [t,T], i.e., there exists a constant i5 > 0 such that 


F{s) ^ SI, a.e. s G [t,T]. 


The following standard assumptions will be in force throughout this paper. 

(HI) The coefficients of the state equation satisfy the following: 

J A(-) e L^(0,T;K"^"), H(-) G T^(0,T;6(-) G L|(H; L1(0,T;R")), 

[ C(-) G L^(0,T; £>(•) G L°“(0,T; a{-) G L^(0,T;R"). 

(H2) The weighting coefficients in the cost functional satisfy the following: 

JggS", Q(-) G T^(0,T;§’"), S'(-) G L2(0,r;R'"^”), R{-) G 

\gGL^^^{n;R^), g(.) G L^(H; 1.1(0,T;R")), p(.) G L^(0,T;R™). 

By [24, Proposition 2.1], under (H1)-(H2), for any {t,x) G [0,T) x R" and u{-) G hl[t,T], the state 
equation (1.1) admits a unique strong solution A(-) = X{- ;t,x,u{-)), and the cost functional (1.2) is well- 
defined. Then Problem (SLQ) makes sense. It is worthy of pointing out that in (H2), we do not impose any 
positive-definiteness/non-negativeness conditions on the weighting matrix/matrix-valued functions G, Q{-) 
and R{-). We now introduce the following definition. 

Definition 2.1. (i) Problem (SLQ) is said to be finite at initial pair {t,x) G [0,T] x R" if 


( 2 . 1 ) 


V(t, x) > —oo. 
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Problem (SLQ) is said to be finite att & [0, T] if (2.1) holds for all x G K", and Problem (SLQ) is said to be 
finite if (2.1) holds for all {t^x) G [0,T] x K". 

(ii) An element u*{-) G U[t, T\ is called an open-loop optimal control of Problem (SLQ) for the initial pair 
(t,x) G [0,r] X R" if 

(2.2) J{t,x;u*f))^J{t,x-,u{-)), V'u(-) G T]. 


If an open-loop optimal control (uniquely) exists for ft^x) G [0,r] x M", Problem (SLQ) is said to be 
(uniquely) open-loop solvable at (t,x) G [0,T] x R”. Problem (SLQ) is said to be (uniquely) open-loop 
solvable at t G [0, T) if for the given t, (2.2) holds for all x G R", and Problem (SLQ) is said to be (uniquely) 
open-loop solvable (on [0,T) x R") if it is (uniquely) open-loop solvable at all (t,x) G [0,T) x R". 

(iii) A pair (Q* (■), v* (■)) G L^(t,r;R™^") x U[t,T] is called a closed-loop optimal strategy of Problem 
(SLQ) on [t,T] if 

(2.3) J(t,x-,e*(-)X*(-) + v*(-)) ^J(t,x-,u(-)), V^tGR”, u(-)GU[t,T], 


where X*(-) is the strong solution to the following closed-loop system: 


(2.4) 


/ 

dX*(s) = { [A(s) + S(s)0*(s)]X*(s) + B(s)v*(s) + 6(s)}ds 

+ { [C(s) + D(s)e*(s)]X*(s) + D(s)v*(s) + (T(s)}dfP(s), 

X*(t)=x. 


s G [t,r], 


If a closed-loop optimal strategy (uniquely) exists on [t, T], Problem (SLQ) is said to be (uniquely) closed-loop 
solvable on [t,T]. Problem (SLQ) is said to be (uniquely) closed-loop solvable if it is (uniquely) closed-loop 
solvable on any [t,T]. 


We emphasize that in the definition of closed-loop optimal strategy, (2.3) has to be true for all x G R". 
One sees that if (Q*(-),v*(-)) is a closed-loop optimal strategy of problem (SLQ) on [t,T], then the outcome 
u*(-) = 0*(-)A*(-) -I- v*(-) is an open-loop optimal control of Problem (SLQ) for the initial pair (t,X*(t)). 
Hence, the existence of closed-loop optimal strategies implies the existence of open-loop optimal controls. 
But, the existence of open-loop optimal controls does not necessarily imply the existence of a closed-loop 
optimal strategy. Here is such an example which is inspired by an example for deterministic LQ problems 
found in [29]. 

Example 2.2. Consider the following Problem (SLQ)° with one-dimensional state equation: 


dX(s) = [ui(s) -I- U 2 (s)]ds -\- [mi(s) - U 2 (s)]dW(s), 
X(t) = X, 


and cost functional 


s G [t,l]. 


(2.6) J°(t,X]u(-)) =EX(1)^. 

In this example, u(-) = (ui(-),U 2 (-))^■ It is clear that 

V^(t,x) = inf J^(t,x;u(-)) ^ 0, V(t, a:) G [0,1] x R. 

u{-)eu[t,T] 

On the other hand, for any t G [0,1), /3 ^ by taking 

u^(s) = -^l[*,,+ .](s)Q, sG[t,l], 


5 




we have 

X(s)=0, s G [i + l//3,l]. 

Hence, 

J°(t, X] u^i')) = 0, (t, x) G [0,1) X M. 

This implies that {u^{-) \ /3 ^ i^} ^ family of open-loop optimal controls of the corresponding Problem 

(SLQ)° for the initial pair {t,x) G [0,1) x R, and therefore, 


V°{t,x) 


0, (t, x) G [0,1) X R, 

x^, t = 1, X € R, 


which is discontinuous at t = 1, x 7 ^ 0. Note also that if we take /3 = then the corresponding open-loop 
optimal control, denoted by it(-), is given by 


(2.7) 


u(s) 


X 

2(1-t) 



s G [t, 1], 


which is a constant vector (only depends on the initial pair (t, x)). Thus, it is continuous (in s G [t, 1]). Now, 
we claim that this Problem (SLQ)° is not closed-loop solvable on any [t, 1] with t G [0,1). In fact, if for some 
t G [0,1), there exists a closed-loop optimal strategy (0*(-),x*(-)), then by definition, one must have 


( 2 . 8 ) 


0^J°it,x]e*i-)X*{-)+v*i-)) ^ J°(t,x;M^(-)) = 0 , VxGR. 


Let 




Then we have from (2.8) that the solution X*(-) of the following closed-loop 


system: 


/ 

dX*{s) = { [e^s) + 02 (s)]X*(s) + K(s) + u;(s)] 

+ { [0t(s) - 0;(s)]X*(s) + K(s) - U 2 *(s)] }dIT(s), s G [t, 1], 

X*{t) =x, 


must satisfy 

Note that 


N*(1)=0, VxGM. 


d[]EX*(s)] = {[0*(s) +0;(s)]EX*(s)+E[uns)+ 1 ^ 2 ( 5 )] }rfs, sG[t,l], 
EX*{t)=x. 


Consequently, 


0 = EX*(1) = + 




which is impossible since the above has to be true for all x G R. Hence, the corresponding Problem (SLQ)° 
is not closed-loop solvable on any [t, 1] with t G [0,1), although the problem admits a continuous open-loop 
optimal control for any initial pair (t,x) G [0,1) x R". 
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Due to the above indicated situation, unlike in [24], and in classical literature on LQ problems, we 
distinguish the notions of open-loop and closed-loop solvabilities for Problem (SLQ). We repeat here that for 
given initial time t £ [0, T), an open-loop optimal control is allowed to depend on the initial state x, whereas, 
a closed-loop optimal strategy is required to be independent of the initial state x. Because of the nature of 
closed-loop strategies, we define the finiteness of Problem (SLQ) only in terms of open-loop controls. 

To conclude this section, we present some lemmas which will be used frequently in sequel. 

Lemma 2.3. Let (H1)-(H2) hold and ©(•) £ L^(0,T;R™^"'). Let P(-) £ (^([O,Tj;S") be the solution 
to the following Lyapunov equation: 

{ P + P{A + BQ) + {A + Be)^P -f (C + De)'^P{C + DQ) 

+ e^Re + S^e + Q^S + Q = 0, a.e. s£[0,T], 

P(T) = G. 

Then for any {t,x) £ [0,T) x M” and u(-) £ U[t,T], we have 

J°(t, x; e(-)X(-) + u(-)) = (P(t)x, a;) + E ^ {2( [B^P + PC + S+{R + PD)e] X, u) 

+ {{R + D^PD)u,u)'^ds. 

Proof. For any {t,x) £ [0,r) x R” and u(-) £ U[t,T], let X{-) be the solution of 

J dX{s) = [{A +Be)X + Bu\ds+ [{C + De)X + Du]dW{s), s £ [t,T], 

I X(t) = X. 

Applying Ito’s formula to s i-J- {P{s)X{s),X{s)), we have 

./•(M;e(-)X(.) + = E { ioxm,x(T)) + ^ J . J ) * j 

= {Pipx, x)+eJ I (pat, X) + {P[{A + BQ)X + Bu],X) + (PX, {A + BQ)X + Bu) 

-f (P [(C -1- DQ)X -t Du] ,{C + DQ)X + Du) + {QX, X) 

+ 2{sx, ex + u) + {R{ex + u),ex + u) jds 

= {P{t)x,x) + E J ^^{[P + P{A + Be) + {A + Be)'^P+{C + DO)^P{C + DQ) 

+ Q + 0^5' -f -f 0^P0] A, A) 

-h 2( [B^P + D^PC + S+{R + D^PD)e] X,u) + {{R + D^ PD)u, M)|ds 
= {P{t)x,x) +E ^^2{[B^ P + D^ PC + S + {R + D^ PD)e]X,u) + {{R + D^ PD)u,u)'^ds. 

This completes the proof. □ 

The following lemma is concerned with the solution to a Lyapunov equation, whose proof can be found 
in [10] (see also [30, Chapter 6]). 

Lemma 2.4. Let 

A(-) £ L^(0,T;R"^”), C(-) £ P^(0,T;R"^”), Q(-) £ L^(0,T;§"), G £ S”. 
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Then the following Lyapunov equation 


P(s)+F(s)A(s) + A(s)^P(s) + C(s)^F(s)C(s) + Q(s)=0, a.e. se[0,T], 
F(T) = G, 

admits a unique solution P(-) £ given by 

[$(s)$(t)-i] ^Q(s) [$(s)$(t)-i] ds 

where $(•) is the solution of 


F{t) =E<^ [$(r)$(i)-^]^G[$(T)$(i)- 


J (i$(s) = A(s)$(s)ds + C(s)$(s)(iW^(s), s > 0, 
\ $(0) = I. 


Consequently if 


G ^ 0, Q(s) ^ 0, a.e. sS[0,T], 

then F(-) £ G([0,r];S^). 

Lemma 2.5. For any u(-) £ U[t^T], let be the solution of 

(2 10) I = [Ais)X^-\s) + B{s)uis)]ds+[Gis)X<-^\s) + Dis)uis)]dWis), 

Then for any 0(-) £ L^(i:,T; there exists a constant 7 > 0 such that 

pT pT 

(2.11) E \u{s)-e{s)X^^'>{s)\^ds^yE |M(s)pds, Vu{-) £U[t,T]. 

Jt Jt 


s £ [t, T], 


Proof. Let 0(-) S L^(t,T; R"*^"). Define a bounded linear operator £ : U[t,T] —>• U[t,T] by 

£u = it-0X(“). 


Then £ is bijective and its inverse £ ^ is given by 

£-1u = m + 0X(“), 


where X^'^\-) is the solution of 

/ 

dX(“)(s) = { [A(s) + P(s)0(s)]X(“)(s) + P(s)u(s)}ds 
^ +{[G(s)+D(s)0(s)]X(“)(s) + D(s)w(s)}dlT(s), s£[t,T], 

X(“)(t) = 0. 

By the bounded inverse theorem, £“^ is bounded with norm ||£“^|| > 0. Thus, 


E [ \u{s)\^ds = E [ |(£-i£u)(s)pds ^ [ 

Jt Jt Jt 


|(£u)(s)|^ds 


= ||£-i||E |m(s) - e{s)X^^\s)fds, 


Vm(-) ew[t,r]. 



which implies ( 2 . 11 ) with 7 = ||£ ^|| 

Finally, we state the following extended Schur’s lemma whose proof can be found in [3]. 
Lemma 2.6. Let Q S i? € S'" and S G Then 


( 2 . 12 ) 

if and only if 




^ 0 , 


(2.13) 


Q- R^O, 7^(5') c 7^(i?). 


Note that the third condition in (2.13) is equivalent to the following: 


(2.14) 


{I - RR"^) = 0. 


□ 


3 Representation of the Cost Functional 

In this section, we will present a representation of the cost functional for Problem (SLQ), from which we will 
obtain some basic conditions ensuring the convexity of the cost functional. Convexity of the cost functional 
will play a crucial role in the study of finiteness and open-loop/closed-loop solvability of Problem (SLQ). 
The following proposition is a summary of some relevant results found in [30]. 

Proposition 3.1. Let (H1)-(H2) hold. For any t G [0,T), there exists a bounded self-adjoint linear 
operator M 2 {t) : U\t,T] —>■ U\t,T], a bounded linear operator Mi{t) : R” —> U\t,T], an Mo{t) G §" and 
r't(-) € U[t, T],yt G R", Ct S R such that 


J{t,x-,u{-)) = {M 2 {t)u,u) 2{Mi{t)x,u) {Mo{t)x,x) -j- 2{u, vt) F 2{x,yt) + Ct, 

(3-1) J°(t, x; u{-)) = {M 2 {t)u, u) + 2{Mi{t)x, u) + {Mo{t)x, x), 

V(x,m(-)) GR” xU[t,T]. 

Moreover, let (Xo(-),y(-), Z{-)) be the adapted solution of the following (decoupled) linear FBSDE: 

{ dXo(s) = [A(s)Xo(s) -f B{s)u{s)]ds + [C(s)Xo(s) -I- D{s)u{s)]dW{s), s G [t,T], 
dY(s) = - [A(s)^r(s) -b C'(s)^Z(s) -b Q{s)Xo{s) + 5'(s)^w(s)] ds + Z{s)dW{s), s G [t, T], 
Xo{t) = 0, F(r) = GXoiT). 

Then 

(3.3) (M 2 (t)u(-))(s) = B{s)^Y{s) + D{s)^Z{s) + S{s)Xo{s) + R{s)u{s), s G [t, T]. 

Let (Xq(-),Y{-), Z(-)) be the adapted solution to the following (decoupled) FBSDE: 

{ dXo(s) = A(s)Xo(s)ds + C(s)Xo(s)dW(s), sG [t,T], 
dY(s) = - [A(s)^Yis) + C{sf Z{s) + Q(s)Xois)]ds + Z(s)dW(s), s G [t,T], 
Xoit)=x, Y(T) = GXo{T). 

Then 

i {XIi(t)x)(s) = B{s)^Y(s) + D(s)^ Z(s) + S(s)Xo(s), sG[t,r], 

1 Mo(t)x = E[r(t)]. 
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Also, let {Xo{-),Y{-),Z{-)) be the adapted solution to the following (decoupled) FBSDE: 


f dXo(s) = [A(s)Xo(s) + 6(s)]ds+ [C(s)Xo(s) + a(s)]dVF(s), sS [t,T], 

(3.6) I dY{s) = -[A{s)^Y(s) + C(syZ{s) + Q{s)Xo(s) + q{s)]ds + Z{s)dW{s), s € [t,T], 
ixo(i) = 0, Y{T) = GXo{T)+g. 

Then 

(3.7) i/t(s) =S(s)^y(s)+i^(s)^Z(s) + ,5(s)lo(s)+p(s), s € [t,T]. 


Finally, Mo(-) solves the following Lyapunov equation: 


Me(t) + MQ(t)A(t) + A(t)~^MQ(t) + C(t)^MQ[t)C(t) + Q(t) — 0, t G [0,T], 
Mo(T) = G, 


and it admits the following representation: 

(3.9) Mo(i)=Ej [$(r)$(t)-i]^G[$(T)$(i)-i] 


[$(s)$(i)“^] Q(s)[$(s)$(t)-i]ds 


where <&(•) is the solution to the following SDE for W^^^-valued process: 


(3.10) 


d$(s) = A(s)<i>(s)(is + G(s)<i>(s)(iM^(s), s > 0, 

$( 0 ) =/. 


Proof. Let <&(•) be the solution to (3.10). Then $(s) ^ exists for all s ^ 0 and the following holds: 

J d[$(s)-i] =-$(s)-^[A(s)-G(s)2]ds-$(s)-^C(s)dlT(s), s ^ 0, 

[ $( 0 )-^ = 1 . 

By the variation of constants formula, the solution Ar(.) = X{- ■,t,x,u{-)) of the state equation (1.1) can be 
written as follows: 

X{s) = <i)(s)$(t)“^a; + <i)(s) <i)(r)“^ [B{r) — G{r)D{r)] u(r)dr + D{r)u{r)dW{r) 

+ <i)(s)'|^ <i)(r)“^ [6(r) — G(r)(T(r)] dr + ^ $(r)“^(T(r)dVL(r)|’, s G [t,T]. 

Now, let 

ht(s) = <i)(s) <i)(r)~^ [6(r) — G(r)(T(r)] dr + ^ <i)(r)~^cr(r)dlT(r)| , s£[t,r], 

and define the following operators: For any t G [0,T), (x,u(-)) G M" x U[t,T], 

{ {Ltu){-) = <&(.) $(r)“^ [B{r) — C(r)D{r)]u{r)dr + $(r)“^Zl(r)M(r)dlF(r)|. , 

(rtx)(-) = $(-)$(t)-^x, Ltu = {Ltu){T), ftx = {Ttx){T). 

Clearly, for any t G [0,T), 

Lt:U[t,T]^ X[t,T], Tt ^ X[t,T], Lt :U[t,T] ^ Xt, Ft : M" ^ At 
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are all bounded linear operators, where X[t, T] = Lf{t, T ; R”) and Xt = L/jr^ (0; R”). Then, for any t G [0, T) 
and (x,u(-)) G R” x U[t,T], the corresponding state process X(-) and its terminal value X{T) are given by 

I X{-) = {Ttx){-) + {Ltu){-) + ht{-), 

\x{T) = ftX + LtuXht{T). 

Hence, the cost functional can be written as 


J{t, x] u{-)) = (Ttx + LtU + ht{T)'^,TtX + Ltu + ht{T)'^ + 2(^g,TtX + Ltu + ht{T)'^ 

+ (^Q(TtX + LtU + ht), TtX + LtU + ht) + 2(^S{TtX + LtU + hj), u) 

+ {Ru, u) + 2(g, Ttx + LtU + ht) + 2(p, u). 

In the above, (•, •) is used for inner products in possibly different spaces. Further, the adjoint operators 
L; : X[t, T] U[t, T], r* : X[t, T] -G R”, £* : Xt U[t, T], f* : Tt R” 

are given by the following: 

mO{s) = B{s)^Yo{s) + D{s)^Zo{s), se[t,r], r:^ = E[yo(i)], 

with (yo(’)) ^o(')) being the adapted solution to the following backward stochastic differential equation 
(BSDE, for short): 


(3.11) 

and 


dYo{s) = -[H(s)^yo(s) + C{s)^ Zq{s) + C(s)](is + Zo(s)(iVF(s), s G [t,T], 

lo(T) = 0, 


{L*,g){s) = B{syY^[s)+D{s)^Z^{s), s G [t,T], F^t? = E[yi(t)], 
with (Fi(-)) Zi{-)) being the adapted solution to the following BSDE: 


dYi{s) = —[H(s)^yi(s) + C{s)^Zi{s)]ds + Zi{s)dW{s), s G [t,T], 
YiiT) = r,GXT. 


(3.12) 

Then with the well-defined adjoint operators, we can rewrite the cost functional as follows: 


J(t, x] u{-)) — ^ GLt X QLt -|- SLt X L'^R^u, u) 

X 2{{L*Gft X LlQTt X 5Fi)x, u) + ((F^GFi + TlQTt)x,x) 

+ 2(a:, Fj \Ght{T) X g\ -I- Fj (Qht X q)) 

X 2(u, [Ght(T) X g] X {Qht X q) X Sht X p) 

X (ht{T), Ght{T) X 2g) X {ht, Qht X 2q) 

= {Xl 2 {t)u, u) X 2{Mi{t)x,u) X {Mo{t)x,x) X 2.{x,yt) X 2(u, vt) X Ct, 

with M 2 {t) : U\t,T] — U[t,T\ being bounded and self-adjoint, Mi(t) : R” — U\t,T] being bounded, and 
Mo(t) G S”; yt G R”, vt{-) G U[t,T] and c* G R. Note that vt{-),yt,ct = 0 when h{-),a{-),g,q{-), p{-) = 0. 
This gives (3.1). Further, if we let 

Xtt{-) = {Ltu){-), y(.) = ro(.) + yi(.), z(.) = Zo(-) + ^i(-), 
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(3.14) 


with 

^(•) = Q{-)Xo{-) + SiYui-), r; = GXo{T), 

then Xo(-) satisfies 

f dXo{s) = [yl(s)Xo(s) + B{s)u{s)]ds + [C'(s)Xo(s) + D{s)u{s)]dW{s), s G [t,T], 
\xo(t)=0 , 

and (F(-),Z'(-)) is the adapted solution to the following BSDE: 

I dY(s) = -[A(s)'^y(s) + C(s)^Z(s) + Q(s}Xo(s) + S(s)'^u(s)]ds + Z(s)dW(s), s G [t,T], 
\ YiT) = GXo{T). 

Thus (3.3) follows. If we let 

Xo(-) = (r*x)(-), Y{-) = Yo{-) + Zi{-), Z{-) = Zo{-) + Zi{-), 

with 

a-) = Qi-)Xoi-), ii = GXoiT), 

then Xo(-) satisfies 

I dXo(s) = A(s)Xo(s)ds + C'(s)Xo(s)dlT(s), s G [t,T], 

\ Xo{t) = X, 

and {Y{-),Z{-)) is the adapted solution to the following BSDE: 

j dY(s) = — [A(s)^F(s) + G(s)^Z(s) + Q(s)Xo(s)]ds + Z(s)dW(s), s G [t^T], 

\ Y{T) = GXoiT). 

Thus (3.5) follows. Likewise, if we let 

Xo{-) = ht{-), ?(•) =Fo(-) + ^i(-), Z{-) = Zo{-) + Z,i-), 

with 

e(-) = Q(-)^o(-)+?(■), V = GXoiT)+g, 

then Xo(-) satisfies 

f dXois) = [A(s)Xo(s) +6(s)]ds+ [C'(s)Xo(s) + cr(s)]dlE(s), s G [t,T], 

\ Xoit) = 0, 

and iY{-), Z{-)) is the adapted solution to the following BSDE: 

dY{s) = - [kl(s)^f(s) + Gis)^Zis) + Qis)Xo{s) + g(s)] ds + Z(s)dlE(s), s G [t, T], 
YiT) = GXoiT)+g. 

Thus (3.7) follows. Finally, we know that 


(Mo(t)x, x) = J°(t, x; 0) = E 


(GXoiT), XoiT)) + / {Qis)Xois),Ms))ds 


= E 


(G$(T)$(t) ^x, <l’(r)$(t) ^x) + ((3(s)$(s)<i>(t) ^x, <i>(s)$(t) ^x)ds 


12 






Then (3.9) follows. Also, by Lemma 2.4, we see that Mo(-) is the unique solution of Lyapunov equation 
(3.8). □ 

From the representation of the cost functional, we have the following simple corollary. 

Corollary 3.2. Let (H1)-(H2) hold and t G [0,T) he given. For any x G R",A G M and u(-),v(-) G 
U[t, T], the following holds: 


J{t, x\ u{-) + Xv{-)) = J{t, x\ u(-)) + J°(t, 0; ?;(•)) 

(3.17) ^ 

+ 2A1E J (^B{s)^Y{s) + D{s)^ Z{s) + S{s)X(s) + R{s)u{s) + p{s),v{s))ds, 


where (A(-), !"(•), Z{-)) is the adapted solution to the following (decoupled) linear FBSDE: 


(3.18) 


dX(s) = [A(s)X(s) + B{s)u{s) + 6(s)]ds 

+ [C(s)A(s) + D{s)u{s) + a{s)]dW{s), s G [t,T], 

< dY{s) = - [A(s)'^r (s) + C{s)^Z{s) + Q{s)X{s) + S{s)'^u{s) + g(s)] ds 

+ Z{s)dW{s), sG[t,r], 

^Xit) = x, Y(T) = GXiT)+g. 


Consequently, the map u(-) i—> J{t,x',u{ )) is Frechet differentiable with the Frechet derivative given by 


(3.19) VJ{t, X - m(-))(s) = 2[i?(s)''"y(s) + D{s)~'^Z{s) + S{s)X{s) + R{s)u{s) + p(s)], s G [t, T], 


and (3.17) can also be written as 

(3.20) J{t,x;u{-) + Xv{-)) = J{t,x-,u{-)) + X'^j'^{t,0;v{-)) + XE (VJ[t, x;u{-j){s), v{s))ds. 


Proof. From Proposition 3.1, we have 
J{t,x;u{-) + Xv{-)) 

= {M 2 {t){u + An), u + An) + 2(^Mi{t)x, u + An) + {Mo{t)x, x) + 2(u + An, vt) + 2(a;, yt) + Ct 
= (M 2 {t)u, u) + 2X{M2{t)u, n) + A^(M 2 (t)n, n) + 2(Mi{t)x, u) + 2X(Mi{t)x, n) + (Mo(t)x, x) 

+ 2(u, vf) + 2A(n, vf) + 2{x, yt) + Ct 
= J{t,x;u{-)) + A^ J°(t,0; n(-)) + 2X{M2{t)u + Mi{t)x + z^t,n). 

From the representation of M 2 {t) and Vt in Proposition 3.1, we see that 

{M 2 {t)u){s) + {XIi{t)x){s) + vt{s) = B(sy'Y{s) + D{s)^Z{s) + 5'(s)A(s) + R{s)u(s) + p{s), s G [t,T], 

with {X{-),Y (•), Z{-)) being the adapted solution to the FBSDE (3.18). The rest of the proof is clear. □ 

Note that if u{-) happens to be an open-loop optimal control of Problem (SLQ), then the following 
stationarity condition holds: 

(3.21) DJ(t,a:;n(-)) = 2[B(s)Ty(s)+D(s)’^Z(s)-bS’(s)A(s)-hi?(s)n(s)-bp(s)] =0, s G [t,T], 

which brings a coupling into the FBSDE (3.18). We call (3.18), together with the stationarity condition 

(3.21) , the optimality system for the open-loop optimal control of Problem (SLQ). 
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The following is concerned with the convexity of the cost functional, whose proof is straightforward, by 
making use of the representation (3.1) of the cost functional. 

Corollary 3.3. Let (H1)-(H2) hold and let t € [0,r) be given. Then the following are equivalent: 

(i) u{-) !->■ J{t,x-,u{-)) is convex, for some x G R”. 

(ii) u(-) 1 -^ J{t,x;u{-)) is convex, for any x G R”. 

(hi) m(-) !->■ J°(t, x; it(-)) is convex, for some x G R". 

(iv) m(-) J°{t, x', u{-)) is convex, for any x G R". 

(v) J°(t, 0;m(-)) > 0, for all u{-) G W[t,T]. 

(vi) M 2 (t) > 0 . 

Similar to the above, we have that u(-) i—> J{t,x',u{-)) is uniformly convex if and only if 

(3.22) J°{t,0;u{-))^ XE \u{s)\^ds, Vit(-) G W[t, T], 

for some A > 0. This is also equivalent to the following: 


(3.23) M2{t)^XI, 

for some A > 0. Further, it is obvious that if the standard conditions (1.4) hold, then 

(3.24) M2it) = L*GLt + L*{Q - S^R-^S)Lt + R-^ + R^){R-^SLt + R^) ^ 0, 

which means that the functional u{-) i—>■ J°(t, 0, «(•)) is convex. The following result tells us that under (1.4), 
one actually has the uniform convexity of the cost functional. 

Proposition 3.4. Let (H1)~(H2) and (1.4) hold. Then for any t G [0,T), the map u{-) i—J°(t, 0; it(-)) 
is uniformly convex. 

Proof. For any u(-) G U\t,T], let Ali“i(-) be the solution of 

= [^(s)X(“)(s) + B(s)u(s)]ds+ [C(s)x(“i(s) + D(s)u(s)]dlF(s), s G [t,T], 

X^^\t) = 0. 

Then by Lemma 2.5 (taking ©(•) = — R{-)~^S{■)), we have 


rT ^ 


J°{t, 0; u{-)) = E (GX(“i(T), X(“)(T)) + / (QAT^"), + 2{SX^^\u) + {Ru, u) 




ds 


rT , 


^ E 


= E 


(QA:(“),X("i) +2(S'A:(“i,u) + {Ru,u) 


ds 


fiSE \u + R-^SX^'^'>\^ds^SyE \u{s)\^ds, Vm(-) G G[t, T], 

Jt Jt 


ds 


for some 7 > 0. This completes the proof. 
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4 Solvabilities of Problem (SLQ), Uniform Convexity of the Cost 
Functional, and the Riccati Equation 

We begin with a simple result concerning the open-loop solvability of Problem (SLQ). 

Proposition 4.1. Let (H1)-(H2) hold. Suppose the map u(-) >—>■ J°(0,0; u(-)) is uniformly convex. Then 
Problem (SLQ) is uniquely open-loop solvable, and there exists a constant a € R such that 

(4.1) V'^{t,x) ^ a\x\^, V(t,x) G [0,T] X R”. 


Note that in the above, the constant a does not have to be nonnegative. 

Proof. First of all, by the uniform convexity of m(-) i—>■ J*^(0,0; «(•)), we may assume that 

(4.2) J°(0,0;m(-)) > AE / |u(s)pds, Vu(-) G W[0,T], 

Jo 

for some A > 0. Now, for any t G [0,T), and any u(-) G U[t,T], we define the zero-extension of u(-) as follows: 

0, sG[0,t), 


(4.3) 


[0/[o.t) ® «(•)](«) = 


^u(s), s€[t,T]. 

Then v{-) = 0J[o,t) ® u(-) G U[0,T], and due to the initial state being 0, the solution Af(s) of 

dX{s) = [A(s)Af(s) -I- B{s)v{s)]ds -\- [C(s)Af(s) -I- D{s)v{s)]dW{s), s G [0,T], 

X(0)=0, 


satisfies 


Hence, 


X(s) = 0, sG[0,t]. 


I-T i-T 

(4.4) J°(i, 0; u(-)) = J°(0,0; 0/[o,t) ® u(-)) ^ AEy | [0/[o,t) ® u(-)](s)|^ds = AEy |M(s)p(is. 

Thus, u{-) H> J°{t,x;u{-)) is uniformly convex for any given (t,x) G [0,r) x R". By Corollary 3.2, we have 

J{t,x;u{-)) = J{t, x; 0) -I- J°(i, 0; u(-)) -\-K {VJ[t, x; 0)(s), u{s)')ds 
(4-5) ^ J(i, x; 0)-I-J°(t, 0; u(-)) —-^E f 1^(3)1^615 — ;^E f \VJ{t,x-0){s)\^ds 

2 Jt 2 A Jt 

>-^E f |M(s)pds-I-T(i, x; 0) — —E f \'DJ{t,x;0){s)\^ds, Vm(-) G W[t,T]. 

2 Jt 2 A 

Consequently, by a standard argument involving minimizing sequence and locally weak compactness of 
Hilbert spaces, we see that for any given initial pair (t,x) G [0,T) x R", Problem (SLQ) admits a unique 
open-loop optimal control. Moreover, when b{-), a{-), g,q{-), p{-) = 0, (4.5) implies that 


(4.6) 


1 

P°(t, x) ^ J°(t, x; 0) — —E J |T>J°(t, x; 0)(s)pds. 


Note that the functions on the right-hand side of (4.6) are quadratic in x and continuous in t. (4.1) follows 
immediately. □ 
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Now, we introduce the following Riccati equation associated with Problem (SLQ): 


' P{s) + P{s)A{s) + A{s)^P{s) + C{s)^P{s)C{s) + Q{s) 

(4 7) , -[P{s)B{s)+C{syP{s)D{s) + S{sy][R{s) + D{syP{s)D{s)]^ 

■[B{syP(s)+D{s)^P{s)C(s)+S{s)]=0, a.e. sG[0,r], 

. P(T) = G. 

A solution P(-) G C'([0,T];S") of (4.7) is said to be regular if 

(4.8) n{B{s)^P{s)+D{syP{s)C{s) +S{s)) <zn{R{s)+D{syP{s)D{s)), a.e. sG[0,T], 


(4.9) [R{-) + D{-y P{-)D{-)]^ [B{-)^ P{-) + D{-y P{-)C{-) + S{-)] G L2(o, T; 
and 

(4.10) R{s) + D[syP{s)D{s) ^ 0, a.e. sG[0,r]. 

A solution P(-) of (4.7) is said to be strongly regular if 

(4.11) R{s) + D{s)^P{s)D{s) ^ XI, a.e. sG[0,r], 


for some A > 0. The Riccati equation (4.7) is said to be (strongly) regularly solvable, if it admits a (strongly) 
regular solution. Clearly, condition (4.11) implies (4.8)-(4.10). Thus, a strongly regular solution P(-) must 
be regular. Moreover, it was shown in [24] that if a regular solution of (4.7) exists, it must be unique. 

In [1], it was showed that for Problem (SLQ)°, the existence of a continuous open-loop optimal control 
for any initial pair {t,x) G [0,r] x M" is equivalent to the solvability of the corresponding Riccati equation 
(4.7) with constraints (4.8) and (4.10). More precisely, their result can be stated as follows (in terms of our 
notations and equation numbers): 

‘‘‘‘Theorem 4.2. Suppose that B{-),C(-),D{-),Q{-),R{-) are continuous and S(-) = 0. Then Problem 
(SLQ)*^ has a continuous open-loop optimal control for any initial pair (t,x) G [0,r] x R" if and only if the 
Riccati equation (4.7) has a solution P(-) such that (4.8) and (4.10) hold.’’’ 

This result is incorrect. Here is a simple counterexample. 

Example 4.3. Consider the following deterministic one-dimensional state equation: 


dX{s) = u{s)ds, s G \t, 1], 
X{t) = X, 


and cost functional 

J(t,x‘,u(-)) = A(1)2 -h 

The Riccati equation for the above problem reads 


s^u(s)^ds. 


{ PitP 

P(t) = —^l( 04 ](t), a.e. t G [0,1] 

P(l) = l. 

It is easy to see that P(t) = t is the unique solution of (4.12), satisfying (4.8) and (4.10). Now, we claim 
that this problem does not admit an open-loop optimal control for initial pair (0,x) with x ^ 0. In fact, if 
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for some a; ^ 0, there exists an open-loop optimal control u*{-) € Z//[0,T], then by the maximum principle, 
the solution F*(-)) of the following (decoupled) forward-backward differential equation: 


(4.13) 

must satisfy 

(4.14) 


X*is) = u*is), Y*{s) = 0, se[0,l], 

X*(0) = a;, y*(l)=X*(l), 


Y*{s) + s'^u*{s) = 0, a.e. se[0,l]. 


Observe that the solution (X*(-), y*(-)) of (4.13) is given by 

X*{s)=x+ [ u*{r)dr, Y*{s) = X*{l), sG[0,1]. 

Jo 

Hence, 

u*{s) = a.e. sG(0 , 1]. 

Noting that u*{-) is square-integrable, we must have X*(l) = 0 and hence u*{-) = 0. Consequently, 

a; = X*(l)— f u*{r)dr = 0, 

Jo 

which is a contradiction. 

From the above example, we see that the sufficiency part of the above Theorem 4.2 (a result from [1]) 
does not hold. We will see in Section 7 that the necessity part does not hold either. 

Instead of Theorem 4.2, in [24], the following were proved, which establishes the equivalence between the 
closed-loop solvability of Problem (SLQ) and the regular solvability of the Riccati equation (4.7). 

Theorem 4.4. Let (H1)-(H2) hold. Problem (SLQ) is closed-loop solvable on [0,T] if and only if the 
Riccati equation (4.7) admits a regular solution P(-) G C([0,r];S”) and the adapted solution (?7(-)iC(’)) of 
the following BSDE: 


(4.15) 


dq{s) = {PB+ C^PD +S^){R +D^PD)^B^]t] 

+ [C^- {PB + C^PDP S^){RP D^PD)'^D^]C 
< -f[C^- {PB + C^PD P S^){RP D^PD)"^D^]P(t 

-{PB + C^PD + S^){RP D^PD)"^p + Pb + q^ds + C,dW{s), s G [0,r], 

. ri{T) = g, 


satisfies 


(4.16) 


B^rj^Pap G TZ{RPD), a.e. a.s. 
{R + PD)^{B^'n + C + P<J + P) e L^(0,r;R'"). 


In this case, Problem (SLQ) is closed-loop solvable on any [t,T], and the closed-loop optimal strategy 
(©*(•), ?;*(•)) admits the following representation: 


e* = -{R +D^PD)HB^P + D^PC + S) + [I - {R +D^PD)HR +D^PD)]U, 

V* = -{R + D^PD)t {B^ri + Pa + p)+[l-{R + PD)"^ (i? -f D~^PD)] v, 
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for some n(-) £ L^(t, T; and i/(-) £ L^{t,T;M.'^), and the value function is given by 


(4.18) 


V(t, x) = e| {P{t)x, x) + 2{r]{t), ^) + + 2(?7, + 2(C, cr) 

- {{R + D^PD)\B^ri + D^C + Pa + p), B^r] + + p) 


ds 


Note that in Example 4.3, the solution P{t) = t to the Riccati equation (4.12) is not regular since 
[i?(t) + D{tyP{t)D{t)]^B{tfP{t) + D{tyP{t)C{t) + S{t)] = il(o,i](t) i L'(0,1;M). 

Hence, by Theorem 4.4, the corresponding LQ problem does not have a closed-loop optimal strategy. 

From the above theorem, we see that the existence of a strongly regular solution to the Riccati equation 
(4.7) implies the unique closed-loop solvability of Problem (SLQ), which, by the remark right after Definition 
2.1, implies the unique open-loop solvability of Problem (SLQ). Particularly, when 6('), cr(-), g, g(-), /o(-) = 0, 
the adapted solution of (4.15) is (0,0), and (4.16) holds automatically. Thus, the existence of a regular 
solution to the Riccati equation (4.7) is equivalent to the closed-loop solvability of Problem (SLQ)°, which 
implies the open-loop solvability of Problem (SLQ)°. However, from Example 2.2, we know that the inverse 
is false, i.e., it may happen that for any initial pair {t,x) £ [0,T) x M", Problem (SLQ)° admits an open- 
loop optimal control (which could even be continuous), while the problem is not closed-loop solvable, which 
means that (4.7) does not admit a regular solution (see Section 7 for further details). On the other hand, 
it is known that under the standard conditions (1.4), the Riccati equation (4.7) admits a unique solution 
P(-) £ C'([0,T];§"), and Problem (SLQ) admits a unique open-loop optimal control which has a state 
feedback form, represented via the solution of the Riccati equation (see [30, 10]). To summarize up, we have 
the following diagram: 


GSsO, R>0, Q-S^R-^S^O 


11 

4 



u{-) 1 —)• uniformly convex 

RE strongly regularly solvable 


RE regularly solvable 

11 

4 


$ 


(SLQ) uniquely open-loop solvable 


(SLQ) uniquely closed-loop solvable 


(SLQ)® closed-loop solvable 


where “RE” stands for the Riccati equation (4.7). It is clear that the uniform convexity of the map u{-) 
J^{t, x; m(-)) does not imply the standard conditions (1-4), which will be even clearer by the results of Section 
5 below. Therefore, it is a desire to establish the following: 

u{-) I— J^{t,x;u{-)) uniformly convex 

This is our next goal. To achieve this, we first present the following proposition, which plays a key technical 
role in this section. 

Proposition 4.5. Let (H1)-(H2) and (4.2) hold. Then for any 0(-) £ the solution 

P(-) £ C'([0,T];S") to the following Lyapunov equation: 


RE strongly regularly solvable 


{ P + P{A + BQ) + {A + BQ)^P +{C + DQ)^P{C + DQ) 
+ e^Re + s^e + e^s + Q = o, a.e. s£[o,t], 
P{T) = G, 


18 













satisfies 


(4.20) R{t) + D{t)^P{t)D{t) ^ XI, a.e. te[0,T], and P(t)-^al, VtG[0,r], 

where a G R is the constant appears in (4.1). 

Proof. Let ©(•) G L^(0,T; R'"^") and let P(-) be the solution to (4.19). For any u(-) G U[t),T], let Xo(-) 
be the solution of 

dXo(s) = [{A +BQ)Xo +Bu]ds+ [{C P DQ)Xq + Du\dW{s), s G [0,T], 

^o(O) = 0. 

By (4.2) and Lemma 2.3, we have 

AE [ |0(s)Xo(s) + u(s)|2ds^ J°(O,O;0 (-)Xo(-)+m(-)) 

Jo 

= E ^ |2( [B^P + D^PC + S + {R + D^PD)e] Xo,u) + {{R + PD)u, u) jfis. 

Hence, for any u{-) G W[0,T], the following holds: 


E 


(4.21) 


^ |2( [B^P + D^PC + S+[R + D^PD - A/)0] Xq, u) 

+ {{R + PD - XI)u,u)^ds = |0(s)Xo(s)|^ds ^ 0. 


Now, fix any uq G R™, take u{s) = wol[t,t+/t] (s); with 0^t<t + h^T. Then 

fi[EXo(s)] = I [H(s) + H(s)0(s)]EXo(s) + H(s)Mol[t.t+?i](s)|ds, sG [0,T], 
EXo(O) = 0. 


Hence, 


EXo(s) = 


0 , 

$(s) 


s G [0, t]. 




$(r) '^B{r)uodr, sG[t,r], 


where <!)(•) is the solution of the following R"^"-valued ordinary differential equation: 

$(s) = [H(s) + H(s)0(s)]$(s), s G [0,T], 

$(0) = I. 

Consequently, (4.21) becomes 

nS 

|2( [B^P P D^PC PSp{RP D^PD - A/)0] 4>(s) J B{r)uodr, uq) 

p (^{Rp PD — XI)uo, Uq) jets ^ 0. 

Dividing both sides of the above by h and letting h 0, we obtain 

([i?(t)-f D(t)’^P(t)D(t) - A/]mo,uo) ^ 0, a.e. t G [0,T], Vmq G R™. 
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The first inequality in (4.20) follows. To prove the second, for any (t,x) S [0,T) x R” and u(-) G U[t,T], let 
X(-) be the solution of 

dX{s) = [{A + B&)X + Bu]ds+ [{C + DQ)X + Du\dW{s), s G [t,r], 

X{t) = X. 

By Proposition 4.1 and Lemma 2.3, we have 

a\x\^ ^ V°{t,x) ^ J°(f, x; 0(-)X(-) +it(')) 

= {P{t)x,x)+E ^^2{[B'^P+D^PC+S+{R+D^PD)e]X,u) + {{R+D^PD)u,u)'^ds. 

In particular, by taking u(-) = 0 in the above, we obtain 

{P(t)x,x) > q;|x|^, V(t, x) G [0,T] X R", 

and the second inequality therefore follows. □ 

Now we state the main result of this section. 

Theorem 4.6. Let (H1)-(H2) hold. Then the following statements are equivalent: 

(i) The map u{-) J°(0,0;u(-)) is uniformly convex, i.e., there exists a A > 0 such that (4.2) holds. 

(ii) The Riccati equation (4.7) admits a strongly regular solution P(-) G (^([O, T]; S"). 

Proof, (i) ^ (ii). Let Pq be the solution of 


(4.22) 


Po + PoA + A^Po + C^PoC + Q = 0, a.e. s G [0, T], 
Po{T)=G. 


Applying Proposition 4.5 with 0 = 0, we obtain that 

R{t) + D{t)^PQ{t)D{t) ^ XI, Po{t) > al, a.e. t G [0,T]. 

Next, inductively, for i = 0,1, 2, • • •, we set 

(4.23) e^ = -{R + D^P,D)-\B^P^ + D^P^C + S), A, = A + P0„ a = C + De^, 

and let Pi+i be the solution of 


(4.24) 


Pj+i + Pi+iAi + AJ Pi+i + Pi+iCi + 0 ^RQi + Qi + 0 ^S Q — 0, a.e. s G [0, T], 

p,+i(r) = G. 


By Proposition 4.5, we see that 

(4.25) P(t) + P(t)^Pi+i(t)P(t) ^ A/, Pi+i{t)^al, a.e. sG[ 0,T], i = 0,1,2,' 

We now claim that {Pi}“i converges uniformly in G([0, T]; §"). To show this, let 

\^P,-P,+i, A, 4 0,_i-0,, 
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Then for i ^ 1, we have 


— — Pi+1 — Pi 

= Pi-^i-l + -Aj-iPi + Cj-iPiCi-1 + ^J-iR^i-l + S'^0i-1 + QJ_iS 
(4-26) - P,+iA, - Aj P,+i - C7P.+iC, - ©7i?0, - 5^0, - 075 

= ^iAi + AJ Aj + Cj AiCi + Pj(Aj_i — Ai) + (Aj_i — Aj)^Pj 

+ Cj^iP.Q.i - C7P.C, + 07_ii?0._i - 07i?0, + + Ajs. 


By (4.23), we have the following: 


{a,_i-A,=BA,, a-i-C,=DA,, 

(4-27) I Cj7iP*C,_i - CjP,Q = AjD^P,DA, + CjP,DA, + AjP,C,, 

[ 07_1-R0i-1 - ©7 770* = A7 RA, + Aj i?0j + ©7 RA,. 

Note that 

B^P, + D^P,Ci + Re^ + S = B^P, + D^P,C + S+{R + PiD)Qi = 0. 


Thus, plugging (4.27) into (4.26) yields 


~ (Aj + Aj*4j + AJ Aj + CjAiCi) 

= P,BA, + A7 B^P, + A7 D^P,DA, + Cj P^DA, + A] P,C, 

(4-28) +a7 RA, + Aj i?0, + ©7 RA, + A, + Aj S 

= Aj {R + D^P,D)A, + {P,B + Cj P,D + QJr + S^)A, + Aj {B^ P, + P,C, + i?0, + S) 

= Aj {R + D^P,D)A, ^ 0. 

Noting that Aj(T) = 0 and using Lemma 2.4, also noting (4.25), we obtain 

Pi{s)^R{s)^R+i{s)^aI, Vse[0,T], Vz > 1. 

Therefore, the sequence {Pi}^i is uniformly bounded. Consequently, there exists a constant K > 0 such 
that (noting (4.25)) 


'|P*(s)|, |i?,(s)KiL, 

(4 29) |0*(s)|<iL(|i?(s)| + |C'(s)| + |5(s)|), 

1*4,(s)| ^ |*4(s)| + K\Bis)\{\B{s)\ + |C(s)| + \S{s)\), 
. 1^(5)! ^ |C'(s)| + K{\B{s)\ + |C(s)| + |5(s)|), 

where Ri{s) = R{s) + {s)Pi{s)D{s). Observe that 


a.e. s € [0, T], V i > 0, 


(4.30) 


Aj — 0j_i 0j 

= R-^D^A,_iDR-_\{B^R + RC + S) - R-\{B^A,_i + 77^A,_iC). 


Thus, noting (4.29), one has 


(4.31) 


|A,(s)Ti?,(s)A,(s)| ^ (|0,(s)| + |0,-i(s)|) |i7,(s)| |0,-i(s) - 0,(s)| 

<iL(|i?(s)| + |C'(s)| + |5(s)|)V*-i(s)|. 
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Equation (4.28), together with Ai{T) = 0, implies that 


A,(s)= [ {A,A, + AjA, + CjA,a + AjR,A,)dr. 

J S 

Making use of (4.31) and still noting (4.29), we get 


I A, 


i(s)| ^ / <p(r) |Ai(r)| + |Aj_i(r)| dr. Vs £ [0,T], Vi ^ 1, 

J S ^ 

where Lp{-) is a nonnegative integrable function independent of Ai(-). By Gronwall’s inequality, 
|A,(s)| ^ e^o vir)dr r<^(^)|A,_i(r)|dr = c T (p(r)|A,_i(r)|dr. 

J S J S 

a A max |Ao(s)|. 


Set 


O^ssCT 


By induction, we deduce that 

2 P 

\Ai{s)\ ^ J (p{r)dr^ , Vs G [0,T], 

which implies the uniform convergence of {Pi}^i- We denote P the limit of {Pi}^^, then (noting (4.25)) 

Ris) + D{s)^P{s)D{s) = lim R{s) + D{s)^Pi{s)D{s) ^ XI, a.e. s G [0,T], 

2^00 

and as i —> oo, 

0* ^ -{R + D^PD)-^{B^P + D^PC + S) = e in L^, 

Ai ^ A + BQ in Ci ^ C + D& in L^. 

Therefore, P(-) satisfies the following equation: 


{ P + P{A + BQ) + {A + BQ^P + (C + D&yP{C + D&) 

+ 0^i?0 + S'^0 + 0^5' + Q = O, a.e. s G [0,T], 

P{T) = G, 

which is equivalent to (4.7). 

(ii) => (i). Let P(-) be the strongly regular solution of (4.7). Then there exists a A > 0 such that 
(4.32) P(s) + £>(s)Tp(s)£>(s) > AJ, a.e. sG[0,r]. 


Set 


0 4 -{R + D^PD)-^{B^P + D^PC AS) G 


For any m(-) G 7/[0,T], let A(")(-) be the solution of 


dX(“)(s)= [4 (s)A(“)(s)+P( s)u(s)]ds+ [C{s)X^^\s) + D{s)u{s)]dW{s), sG [0,T], 

A(0) = 0. 
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Applying Ito’s formula to s i—)• (P(s)X(“^(s), A*^“)(s)), we have 


J°(0,0;m(-)) =]Ei(GA(“)(T),X(“)(T)) 


S' R . 




' ds 


= E 


= E 


= E 


= E 


[ [(PX(“),X(“)) + (P(AX(“) +Pu),x(")) + (PX("\AX(“) +Pu) 

Jo *- 

+ {P{CX^^^ + Du ), GX(“) + Du) + (QX(“\ x(")) + 2(SX("\ w) + (Pu, w)j ds 

[ [((P + PA + A^P + G^PG + Q)X(“\ X(")) + 2((P^P + P^PG + S)x(“\ u) 

Jo 


+ ((P + pTpP)M,u) 


ds 


(0^ (P + D^PD) 0X(“), X(“)) - 2( (P + D^PD) 0X(“), m) + ((P + P^PP)m, m) 

[ ((P + pTpP)(u-0X(“)),M-0X(“))ds. 

Jo 


ds 


Noting (4.32) and making use of Lemma 2.5, we obtain that 

J°(0,0;u(-)) =E / ((P + P^PP)(it-0X(“)),u-0X(“))ds ^ AyE / |u(s)pds, Vu(-) £ G[0,T], 

Jo Jo 

for some 7 > 0. Then (i) holds. □ 

Remark 4.7. From the first part of the proof of Theorem 4.6, we see that if (4.2) holds, then the 
strongly regular solution of (4.7) satisfies (4.11) with the same constant A > 0. 

Combining Theorem 4.4 and Theorem 4.6, we obtain the following corollary. 

Corollary 4.8. Let (H1)-(H2) and (4.2) hold. Then Problem (SLQ) is uniquely open-loop solvable at 
any {t,x) £ [0,P) x R” with the open-loop optimal control u*{-) being of a state feedback form: 

(4.33) u*i-) = -(P + P^PP)-i(P^P + P^PG + S)X* - (P + D^PD)-\B^r] + P^C + + p), 

where P(-) is the unique strongly regular solution of (4.7) with being the adapted solution of 

(4.15) and X*{-) being the solution of the following closed-loop system: 

dX*{s) = |[A-P(P + P^PP)-^(P^P + P^PG + S)]X* 

- P(P + P^PP)-i(P^r 7 + P^C + P^Pcr + p) + bjds 
(-^■34) +|[G-P(P + P^PP)-i(P^P + P^PG + S)]A* 

- D{R + D^PD)-\B^p + D^C + D^Pa + p) + a'^dWis), s £ [i,r], 

X*{t) = X. 


Proof. By Theorem 4.6, the Riccati equation (4.7) admits a unique strongly regular solution P(-) £ 
G([0,r];S"). Hence, the adapted solution (??(•): C(’)) of (4A5) satishes (4.16) automatically. Now applying 
Theorem 4.4 and noting the remark right after Definition 2.1, we get the desired result. □ 

Remark 4.9. Under the assumptions of Corollary 4.8, when b{-),a{-),g,q{-),p{-) = 0, the adapted 
solution of (4.15) is (??(•): C(')) = (OjO)- Thus, for Problem (SLQ)°, the unique optimal control u*{-) at 
initial pair (t, x) £ [0, T) x R" is given by 

u*{-) = -{R + D^PD)-^{B^P + D^PC + S)X*, 


(4.35) 
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with P(-) being the unique strongly regular solution of (4.7) and ^*(-) being the solution of the following 
closed-loop system: 


{ dX*{s) = [A - B{R + PD)-^{B^P + PC + S)]X*ds 

+ [C - D{R + PD)-\B^P + PC + S)]X*dW{s), s€ [t,T], 

X*{t)=x. 

Moreover, by (4.18), the value function of Problem (SLQ)° is given by 
(4.37) V°(t, x) = {P(t)x, x), (t, x) e [0, T] x R". 


5 Finiteness of Problem (SLQ) and Convexity of Cost Functional 


We have seen that the uniform convexity of the cost functional implies the open-loop and closed-loop solv¬ 
abilities of Problem (SLQ). We expect that the finiteness of Problem (SLQ) should be closely related to the 
convexity of the cost functional. A main purpose of this section is to make this clear. Other relevant issues 
will also be discussed. First, we introduce the following: 


(5.1) 


A(s,P(.)) = 


P(s)+P(s)A(s)-|-A(s)^P(s)-|-C(s)^P(s)C(s)+Q(s) 
^ B(sC P{s)+D{sC P{s)G{s)+S{s) 


P[s)B{s)+C{8C P{8)D{s)+S{s) 
R{s)+D(sC P(s)D(8) 


for any P(-) € AC{t,T-,S‘^) which is the set of all absolutely continuous functions P : [t,T] —> §". Let 
P[t,T] = {P(-) e AC'(t,T;§") I P(T) ^ G, A(s,P(-)) ^ 0, a.e. s e [t,T]}. 

We have the following result. 

Proposition 5.1. Let (H1)-(H2) hold, and t G [0,T) be given. Among the following statements: 

(i) Problem (SLQ) is finite at t. 

(ii) Problem (SLQ)° is finite at t. 

(hi) There exists a P{t) G S” such that 


(5.2) 


V°{t,x) = {P{t)x,x), Vx S R". 


(iv) The map u{-) >->■ J{t, x; u{-)) is convex, for any a: S R". 

(v) p[t,r]^0. 

the following implications hold: 

(i) (ii) => (hi) => (iv); (v) ^ (ii). 

Proof, (i) => (ii). By Proposition 3.1, for any x G R" and u{-) G h{[t,T], we have 

V(t, x) + V{t, —x) ^ J{t, x; u{-)) + J{t, —x; —u(-)) 

= 2 (M 2 {t)u, u) + 2(Mi(f)x, u) + (Mo(f)x, x) -f c* 
= 2 J°{t,x-,u{-)) + ct , 


which implies (ii). 
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(ii) => (iii) can be shown by a simple adoption of the well-known result in the deterministic case (see 

[11, 4]). 

(iii) (iv). By Corollary 3.3, if it(-) !->■ J(t,x;it(-)) is not convex, then J°(t,0;it(-)) < 0 for some 
u(-) G U[t,T\. By Corollary 3.2, we have 

J°(t, x\ Au(-)) = J°(t, X] 0) + 0; u{-)) + XE (VJ^it, x; 0){s),u{s)')ds, VA G R". 

Letting A —>■ oo, we obtain 

y°(t,x) ^ lim J°(t, x; Au(-)) = —oo, 

A— 

which is a contradiction. 

(v) (ii). For any (t,x) G [0,T) x R", u(-) G W[t,T], and any P(-) G AC'(t,T;§"), one has 

E{PiT)X{T),X{T)) - {P{t)x,x) 

pT 


= E 


P{s) + P{s)A{s) + Gl(s) ' P{s) + (7(5) ' Pis)C{s) X{s), X{s) 


+ 2 


B{s)^P{s) + D{s)^P{s)C{s) X{s),u{s)J -f (D{s)^P{ s)D{s)u{s), u{s)') I’ds. 
Hence, if V[t,T] ^ 0, then by taking P(-) G V[t,T], one has 

’g(s) 5 (s)t\ fx{s)\ fxis)^ 


J\t, X- u{-)) = E I {GX{T),X{T)) + [ 

= (P(t)x,x) -GE|([G-P(T)]X(r),X(r)) + ^"’^A(s,P(.)) , 


S{s) R{s) j \ u{s) j ’ yu{s) j j 

X{s)\ (X{s) 
u{s) 


ds 


' ds 


^ (P(t)x,x), Vu(-) G G[t, T]. 

This implies that the corresponding Problem (SLQ)° is finite at t. □ 

It is worth to point out that the convexity of the map u(-) J^{t, x; «(•)) is not sufficient for the finiteness 

of Problem (SLQ)°. We present the following example (see also [21] for an example of a quadratic functional 
in Hilbert space). 

Example 5.2. Consider the following one-dimensional controlled SDE: 


(5.3) 

and the cost functional: 

(5.4) 

We claim that 

(5.5) 


dX{s) = u{s)ds + X{s)dW{s), sG[t, 1], 
X(t) = X, 


J°(t, x; u{-)) = E 


-X(l)2 + ^ e^-^uisf 


ds 


J°(0,0;u(-)) > 0, Vu(-) GiY[0,P], 


which, by Corollary 3.3, is equivalent to the convexity of «(•) J°(0, x; u(-)), but 

(5.6) H°(0,x) = —oo, Vx ^ 0. 
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To show the above, let u(-) G U[Q,T] and X{-) = X{- ;0,x,u{-)) be the solution of (5.3) with t = 0. By the 
variation of constants formula, 

X(s) / e5’-^Mu(r)dr, se[0,l]. 

Jo 

Taking a: = 0 and noting that jg independent of J>, we have 


E[X(1)2] = E 


nl pl 

'u{r)dr 


,W(l)-lV(r)-i(l-r), 


Jo 


= [ Ee2[^(^)-^(’’)l-(i-’')E[M(r)2]dr = [ e^-^E[u{rf]dr = E [ e^-^u{rfdr, 

Jo Jo Jo 


and hence 


J°(0,0;u(-)) = IE 


—X{lY+ f ®u(s)^c?s 

Jo 


>0, Vu(-) Gi^[0,T]. 


On the other hand, taking x ^ 0 and u{s) = A G R, we have 

X{1) = (x + A)e’^(i)-i 

Therefore, 


J°(0,x;u(-)) =E 


—X{Vf+ / ‘‘u^s^ds 


= -E 


(x + + A^E / 


= —(x + A)^e + A^e = —(x^ + 2Ax)e. 

Letting |A| —>• oo, with Ax > 0, in the above, we obtain T^°(0,x) = —oo. This proves our claim. 

The above example tells us that, besides the convexity of it(-) i—>• J°(t, x; u(-)), one needs some additional 
condition(s) in order to get the finiteness of Problem (SLQ)° at t. To find such a condition, let us make some 
observations. Suppose it(-) i—J°(0,x;u(-)) is convex, which, by Corollary 3.3, is equivalent to the following: 

(5.7) J°(0,0; «(•))> 0, Vm(-) G W[0,T]. 

Then for any e > 0, consider state equation (1.1) (with b{-),a{-) = 0) and the following cost functional: 

<'^I^Q{s) S{sy \(x{s)\ (X{s)^^ ^ 


(5.8) 


J0(t, x; u{-)) 4 E I {GX{T),X{T)) + ^ 


S{s) R{s) + el I \ u{s) I ’ I u{s) 


• ds 


J°(t, x; u(-)) + eE f |M(s)pds. 

Jo 


Denote the corresponding optimal control problem and value function by Problem (SLQ)° and 
respectively. By Corollary 3.3 and the convexity of u(-) i-)- J°(0,0; u(-)), one has 

J°(0,0;u(-)) = J°(0,0; «(•)) +eE / \u{s)fds^eE[ |u(s)|2ds, Vu(-) G W[0, T], 

Jo Jo 

i.e., u(-)^ J°(0,0 ;u(-)) is uniformly convex. Hence, it follows from Theorem 4.6 that the following Riccati 
equation: 


(5.9) 


F, + P,A + A^P, + C^P,C + Q 

-{PeB + C^P^D + S^){R + eI + D^P^D)-\B^P^ + D^P^C + S) = 0, a.e. sG[0,r], 
Pe{T) = G, 
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admits a unique strongly regular solution Pe(-) G such that (noting Remark 4.7) 

(5.10) R(t) +el + D(t)^P^(t)D{t) > el, a.e. t e [0, T]. 

Now, we are ready to state and prove the following result which is a characterization of the finiteness of 
Problem (SLQ)°. 

Theorem 5.3. Let (H1)-(H2) and (5.7) hold. For any e > 0, let Ps(-) be the unique strongly regular 
solution of the Riccati equation (5.9). Then Problem (SLQ)° is finite if and only if {Pe(0)}£>Q is bounded 
from below. In this case, the limit 

(5.11) lim P,(t) = P(t), VtG[0,r], 

€—^0 

exists, and (5.2) holds. Moreover, 

(5.12) R{t) + D(t)^P(t)D{t) ^ 0, a.e. tG[0,r], 
and 


(5.13) N(t) ^ P(t) < Mo(t), Vt G [0,T], 

where Mo(-) is the solution to the Lyapunov equation (3.8), and 


(5.14) N(t) = [$^(t)^] |P(0) - 4 >a(s)^ [C(s)^Mo(s)C(s) + Q(s)J <I>A(s)ds 4>A(t)"\ 

with $a(-) being the solution to the following: 

j 4 >a(s) = ^(s)^a(s), s ^ 0, 


(5.15) 


\ $a(0) 


= I. 


In particular, if Problem (SLQ)° is finite at t = 0, then it is hnite. 


Proof. Necessity. Suppose Problem (SLQ)° is finite and let P : [0,T] S” such that (5.2) holds. For 
any £2 > ei > 0, we have 

J^^(t,x;u{-)) > J°^(t,x;u{-)) > J°{t,x-,u(-)), V(t,a;) G [0,T] x R", Vu(-) G U[t,T]. 

Hence (noting Remark 4.9), 


(Pe, {t)x, x) = 14 ° (t, x) = inf (t, x; u{-)) 

u{‘)^U[t,T\ 

> T] 2;; U(-)) = V;° (t, X) = (Pei (t)x, X) 

^ inf J°(t,x;u(-)) = V°(t,x) = {P(t)x,x), V(t,x) G [0,T] x K”. 

u(-)eM[t,T] 

Thus {P£(t)}£>o is a nondecreasing sequence with lower bound P(t) and therefore has a limit P{t) with 
(5.16) P(t) = limPJt) > P(t), Vt G [0,T]. 

On the other hand, for any (5 > 0, we can find a u^(-) G I4[t, T], such that 

V^{t,x) ^ J°{t,x-,u^{-)) + eM \u^(s)\'^ds ^V'^{t,x) + 6 + eE \u^{s)\'^ds. 
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Letting e —>■ 0, we obtain that 

{P(t)x, x) = lini(Pe(t)a;, x) = lim x) ^ x) + <5 = {P{t)x, x) + <5, 

£—>•0 £^0 

from which we see that 

(5.17) {P(t)x,x) ^ {P{t)x,x) V(t, x) G [0, T] X R". 

Combining (5.16)-(5.17), we obtain (5.11) with P(-) satisfying (5.2). Moreover, letting e —>■ 0 in (5.10), we 
obtain (5.12). 

Sufficiency. Suppose there exists a /3 G R such that 

PPO) ^ /37, Ve > 0, 

then for any x G R" and «(•) G U[0,T], we have 

j”(0,x;«(•)) +eE / \u{s)\^ds ^ V^{0,x) = {Ps{0)x,x) ^ P\x\'^, Ve > 0. 

Jo 

Letting e —^ 0 in the above, we obtain 

J"(0,x;u(-)) ^/3|x|2, VxgR", Vm(-) G W[0,T], 
which implies the finiteness of Problem (SLQ)° at t = 0. 

Now, let P(0) G S” such that P°(0,x) = (P(0)x,x),Vx G R". Then 

(5.18) P(0) < Pe(0), Ve > 0. 

Also, by Remark 4.9 and Proposition 3.1, 

{Pe(t)x, x) = V^(t, x) ^ T°(t, x; 0) = J°(t, x; 0) = (Mo(t)x, x), Vx G R". 

This leads to 


(5.19) P,it)^Mo{t), tG[0,r], Ve>0. 

On the other hand, let 4>^(-) be the solution of (5.15) and set 

He = {PeB + C^PeD + S^){Re + D^P^D)-\B^P^ + D^P^C + S) ^ 0. 
Differentiating s $^(s)^PE(s)<i)A(s), we obtain 


ds - 


^a{s)' PPs)^A{s) =$a(s)' Iips)-C{s)' PPs)Cis)-Q{s) $a(s). 


Hence, combining (5.18)-(5.19), we have 

^ApyPeiP^Ap) = Pe{0) + [ 4'a(s)^ 0^(5) - C'(s)^Pe(s)C'(s) - (^(s) ^A{s)ds 


> 


P(0) - f ^AisV fc'(s)TMo(s)C(s) + Q(s)l $^(s)ds. 
Jo L J 


Thus, 


pyt);? [$A(i)T UPiO)- I 'i>A{sV C{sy Mo{s)C{s) + Q{s) ^A{s)ds\ ^aP)-^ 


(5.20) 


= iV(t), tG[0,r]. 
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Then using the same argument as in the previous paragraph, we can show that 


(5.21) J°{t,x]u{-)) ^ {N{t)x,x), V(t, x) € [0,T] X R", V'u(-) G T], 

which implies the finiteness of Problem (SLQ)°. Moreover, let P : [0,T] ^ S" such that (5.2) holds, then 

{N(t)x,x) ^ inf J^(t,x]u(-)) = (P(t)x,x) ^ J°(t,x;0) = {Mo(t)x,x), V(t, x) G [0, T] x R", 
«(-)ew[t,T] 

and (5.13) follows. 

Finally, if Problem (SLQ)° is finite at t = 0, then (5.18) holds and the finiteness of Problem (SLQ)° 
therefore follows. □ 

The following is another sufficient condition for the finiteness of Problem (SLQ)°, which is a corollary of 
Theorem 5.3 and Proposition 5.1, (v) (ii). 

Corollary 5.4. Let (H1)-(H2) hold. If there exists a A(-) G L^(0,r;S”) such that 

(5.22) R + D^PD^ {B^P + D^PC + S)A-^{PB + C^PD + S^), a.e. sG[0,r], 


where P{-) is the solution of the following Lyapunov equation: 


(5 23 ) J + C(s}^P(s)C(s) + Q(s} = A(s), a.e. s G [0, T], 

\ P(T) ^ G. 

Then Problem (SLQ)° is finite. 

Proof. Under our condition, by Lemma 2.6, one has 


A{s,P{-)) 


( A(s) 

\b(s)'^ P(s) + D(s)'^ P{8)C(s)+S(s) 


P(s)B(s)+C(s)'^ P(s)D{s)+S{s)'^ 
R(s)+D(s)'^ P(s)D{s) 


> 0 , 


a.e. s G [0,T]. 


Hence, P(-) G 7^[0,T] and Problem (SLQ)° is finite at t = 0. Then the finiteness of Problem (SLQ)° follows 
from Theorem 5.3 immediately. □ 

We now return to the study of convexity of the map u(-) >—>■ J^(t,0;u(-)). First, from the definition of 
M 2 {t) (see (3.13)), we see that M 2 {t) ^ 0 if and only if 

(5.24) R{-) ^ - {llGLt + LlQLt + SLt + L^S ^), 

with the right hand side being non-positive. Thus, unlike the well-known situation for the deterministic 
LQ problems (for which i?(-) 0 is necessary for M 2 {t) ^ 0 ([30])), R{-) does not have to be positive 

semi-definite. Actually, as shown by examples in [7, 30], R{-) could even be negative definite in some extent. 
Let us now take a closer look at this issue. 


Note that when u{-) J°(0, 0 ;m(-)) is convex, for any e > 0, the unique strongly regular solution Pe{-) 

to the Riccati equation (5.9) satisfies (5.10) and (5.19). Hence, 

(5.25) R{t) + D(tf Moit)D{t) > 0, a.e. t G [0,T], 


or equivalently. 


(5.26) 


R{t) + U(t)TE| [$(T)$(t)-^] ^G[$(T)$(t)-i] 

-b [ [$(s)$(t)-i]^Q(s)[$(s)$(t)-i]dsli:>(t) ^ 0, a.e. t G [0,r]. 
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This is another necessary condition for the finiteness of Problem (SLQ)°, which is easier to check. From 
(5.26), we see that if R{-) happens to be negative definite, then in order u{-) !->■ J°(0, 0; u{-)) to be convex, it 
is necessary that D{-) is injective, and either G or Q{-) (or both) has to be positive enough to compensate. 
Note that D{s) was assumed to be invertible in [23]. Therefore, in some sense, our result justifies the 
assumption of [23]. 

The following gives a little improvement when more restrictive conditions are assumed. 

Proposition 5.5. Let (H1)-(H2) hold. Suppose that 


(5.27) 


B(-)=0, C(-)=0, S{-)=0. 


Then the map u(-) n- J^(0,0;u(-)) is convex if and only if (5.26) holds. In this case, Problem (SLQ)° is 
closed-loop solvable. 

Proof. It suffices to prove the sufficiency. Note that in the current case, the corresponding Riccati 
equation becomes 


(5.28) 


P{s) + Pis)A{s)+A{syP{s) + Qis) = 0, a.e. se[0,r], 
P{T) = G, 


whose solution is Mq{-). If (5.26) holds, then it is easy to verify that Mq{-) is regular. Consequently, by 
Theorem 4.4, Problem (SLQ)° is closed-loop solvable, and hence u(-) J°(0,0; w(-)) is convex. □ 

Note that in the case (5.27), we have 


Mo{t) = [$A(T)$A(t)-']^G[$^(r)$A(t)-^] + 



[$A(s)^A(t) ^]^(5(s)[$A(s)5>A(t) ^]ds, 


with <I>a(-) being the solution of (5.15). Hence, (5.26) can also be written as 


(5.29) 


R{t) + i^(t)T| [$^(r)$A(t)-^]^G[$A(T)$A(t)-'] 

-b f [$A(s)$A(t)“^]^Q(s)[4>A(s)$yl(t)“^]rfs jG>(t) ^ 0, 


a.e. t G [0 ,T]. 


As we pointed out earlier, Problem (SLQ)° may still be infinite at some initial pair {t, x) even if the cost 
functional is convex. In this case, by Theorem 5.3, the sequence {Pe(t)}£>o diverges. The following result is 
concerned with the divergence speed of {P£(t)}£>o- 

Proposition 5.6. Let (H1)-(H2) and (5.7) hold. For any e > 0, let Pe{-) be the unique strongly regular 
solution of (5.9). Then for any a > 1 and t G [0,r], the sequence {e°‘Pe{t)}e>o converges. 

Proof. Let e > 0 and consider Problem (SLQ)°. By Proposition 3.1 (noting M 2 (t) ^ 0), 


J°{t,x-,u{-)) 


J°(t, x; u(-)) -b eE 



|u(s)p(is 


(\M2{t) -b el]u, u) -b 2 (Mi{t)x, u) -b (Mo(t)x, x) 
[M2 (t) -b e/] 5 It -b [M2 (t) -b e/] ”^ Ml {t)x 


-b {Mo{t)x,x) - {[M 2 {t) -be/] ^Mi{t)x,Mi{t)x). 
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Thus 


(5.30) {Pg{t)x,x) = inf J^{t,x;u{-)) = (Mo{t)x,x) — ([M 2 {t) + el] ^Mi{t)x,Mi{t)x). 

u{-)GU[t,T] 

For any a ^ 1, we have 

0 < e°‘([M 2 {t) + eI]~^Mi(t)x, Mi{t)x) ^ e°'~^(Mi{t)x, Mi(t)x) < (Mi{t)x, Mi{t)x), VO < e < 1. 

Consequently, for any a; G M", {e°‘{Ps{t)x,x)}eyo has a convergent subsequence, and hence the sequence 
{e“(Pe(f)a:, a:)}e>o itself converges as e ^ 0 since it is nondecreasing. The result therefore follows. □ 

From Remark 4.7 and 4.9, we see that if the uniformly convex condition (4.2) holds, then Problem (SLQ)° 
is finite and 


(5.31) 


R{s) + D{s)^P{s)D{s) ^ XI, a.e. sG[0,T], 


where P : [O, T] —S" is the function such that (5.2) holds. The following result shows that the converse is 
also true. 

Theorem 5.7. Let (H1)-(H2) hold. Suppose Problem (SLQ)° is finite and let P : [0,r] —S" such that 
(5.2) holds. If (5.31) holds for some A > 0, then P(-) solves the Riccati equation (4.7). Consequently, the 
map u(-) I—J°(0, 0;m(-)) is uniformly convex. 

Proof. For any e > 0, let Pe(-) be the unique strongly regular solution of (5.9). By Theorem 5.3, 

Pe{t)\P{t), as e\0, Vt€[0,r]. 

Note that Pd') ^ Mo{-) Vs > 0 and by (5.13), P(-) is bounded. Thus, {Pe(t)}£>o is uniformly bounded. 
Also, we have 

P(s) + D{sdPds)D{s) > R{s) + D{sdP{s)D{.s) ^ XI, a.e. s G [0, T], Ve > 0. 

Then it follows from the dominated convergence theorem that 

PeA + A'^Pe + C^PeC + Q- {PeB + PeD + S^){R + sI + P^P^P)"^ (P^P, + P^C + S) = Ae 
converges to 

PA + A^P + C^PC + Q- {PB + C^PD + S^){R + PUy^B^P + PC + S) = A 

in as s ^ 0. Therefore, 

Pit) = lim Pelt) = G + lim [ Ae(s)ds = G + [ A{s)ds, 

e^o Jt 

which, together with (5.31), implies that P(-) is a strongly regular solution of (4.7). Consequently, by 
Theorem 4.6, u{-) i—>■ J°(0, 0 ;u(-)) is uniformly convex. □ 


We now look at the following case: 


(5.32) 


P(.) = 0, P(-)»0. 


Note that although P(-) = 0, since C'(-) is not necessarily zero, our state equation is still an SDE. For such 
a case, the above results can be restated as follows. 
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Theorem 5.8. Let (H1)-(H2) and (5.32) hold. Then the following statements are equivalent: 

(i) Problem (SLQ) is finite at t = 0; 

(ii) Problem (SLQ)° is finite at t = 0; 

(iii) The map u{-) i—J°(0,0 ;m(-)) is uniformly convex; 

(iv) The following Riccati equation 

\ P + PA + A^P + C^PC + Q-{PB + S^)R-\B^P + S)=0, a.e. se[0,T], 
53) < 

P(T) = G, 


admits a unique solution P(-) G G([0, T]; S"); 

(v) Problem (SLQ) is uniquely closed-loop solvable; 

(vi) Problem (SLQ) is uniquely open-loop solvable. 

Proof, (i) => (ii) follows from Proposition 5.1. 

(ii) (iii). By Theorem 5.3, Problem (SLQ)° is finite. Since D{-) = 0,R{-) » 0, (5.31) holds for some 
A > 0, and the result follows from Theorem 5.7. 

(iii) (iv). In the case of (5.32), the corresponding Riccati equation becomes (5.33). If P{-) G 
G([0,T];§”) is a solution of (5.33), then it is automatically strongly regular. Thus, by Theorem 4.6, we 
obtain the equivalence of (iii) and (iv). 

(iii) ^ (vi) follows from Proposition 4.1 and (iv) => (v) follows from Theorem 4.4. 

Finally, (v) ^ (i) and (vi) =J> (i) are trivial. □ 

An interesting point of the above is that under condition (5.32), finiteness of Problem (SLQ) implies the 
closed-loop solvability of Problem (SLQ). In the deterministic case, such a fact was firstly revealed in [31] 
for two-person zero-sum differential games, and was proved in [29] for deterministic LQ problems by means 
of Fredholm operators. 


6 Minimizing Sequences and Open-Loop Solvabilities 


In Section 4, we showed that under the uniform convexity condition (4.2), Problem (SLQ) is open-loop 
solvable and the open-loop optimal control has a linear state feedback representation. In this section, we 
study the open-loop solvability of Problem (SLQ) without the the uniform convexity condition. 

First we construct a minimizing sequence for Problem (SLQ) when it is finite. 

Theorem 6.1. Let (H1)-(H2) hold. Suppose Problem (SLQ) is finite. For any e > 0, let Pe(-) be the 
unique strongly regular solution to the Riccati equation (5.9). Further, let (rje (■), Q (■)) and Xg(-) = Xg(- ; t, x) 
be the (adapted) solutions to the following BSDE and closed-loop system, respectively: 


( 6 . 1 ) 


dpsis) — — {A + BQg)^ rjg F (C H- C,g F (C F P^a — 0jp -I- P^b F q 

FCedWis), sG[0,r], 

PeiT) = g, 


ds 
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(6.2) 

1 dX,(s) = 

(A -\- BQg)Xg -\- Bvg -\- b 

ds 

(C -\- DQg)Xg -\- Dvg -\- u 


\ Xs(t) = X, 





s € [t,T], 


where 

(6.3) 


= -{R + el + D^P^D) {B^P^ + D^PeC + S), 

, - 1 , 


V, = -{R + eI + D^P,D) \B^r,, + D^C, + D^Pea + p). 

Then 

(6.4) Me(-) = 0e(-)^e(-) + ^^.(-), £>0 

is a minimizing sequence of u(-) J(t, x; «(•))• 

(6.5) lim J(t,x;Ue(-)) = inf J{t,x]u{-)) = V{t,x). 

£->o u{-)eu[t,T] 


Proof. For any e > 0, consider state equation (1.1) and the following cost functional: 


( 6 . 6 ) 


Jg{t, X] u{-)) = J{t, X] u{-)) + eE^ |u(s)pds. 


Denote the above problem by Problem (SLQ)^ and the corresponding value function by 14(-, •). By Corollary 
4.8, Ue(-) defined by (6.4) is the unique optimal control of Problem (SLQ)^ at {t,x) G [0,r) x R”. Note that 


eE 


\Ue{s)\'^ds = Je{t,x;ue{-)) - J{t, X] Ue{-)) = Ve{t,x) - J{t,x;Ue{-)) 
^Ve{t,x) — V{t,x) ^ 0 as £—>-0. 


Thus, 


lim J(t, x; Ue(-)) = lim 
£—^0 £—^0 


Ve(t, x) — sE 



|Me(s)|^C?S 


V(t,x). 


The proof is completed. □ 

Using the minimizing sequence constructed in Theorem 6.1, the open-loop solvability of Problem (SLQ) 
can be characterized as follows. 

Theorem 6.2. Let (H1)-(H2) hold. Suppose u(-) i-J- J°(0,0 ;m(-)) is convex. Let (t,x) G [0,r) x R” and 
{ue(-)}e>o he the sequence defined by (6.4). Then the following statements are equivalent: 

(i) Problem (SLQ) is open-loop solvable at (t,x); 

(ii) The sequence {u£(-)}e>o admits a weakly convergent subsequence; 

(hi) The sequence {ue(-)}e>o admits a strongly convergent subsequence. 

In this case, the weak (strong) limit of any weakly (strongly) convergent subsequence of {u£(-)}e>o is an 
open-loop optimal control of Problem (SLQ) at (t,x). 

To prove Theorem 6.2, we need the following lemma. 

Lemma 6.3. Let TL he a Hilbert space with norm || • || and 0,G H, n = 1, 2, • • •. 

(i) If 9n ^ 9 weakly, then ||0|| ^ lim ||0n||- 

n—^oc 
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(ii) On ^ 0 strongly if and only if 


ll^nll ll^ll and On ^ 0 weakly. 


Proof, (i) By the Hahn-Banach theorem, we can choose a 0* S 7^ with ||0*|| = 1 such that {0*,0) = ||0||. 
Thus, 

||0|| = {0*,O) = lim {0*,0n) lim ||0„||. 

n—¥oo ji — 

(ii) The necessity is obvious. Now suppose ||0„|| —||0|| and ^ 0 weakly. Then 
ll^n~^|| = ||^n|P~2(^0,d„^ + ||0||^—^0 as n—>- 00 . 

This completes the proof. □ 

Proof of Theorem 6.2. (i) (ii), and (i) ^ (hi). Let f*(') be an open-loop optimal control of Problem 

(SLQ) at {t,x). By Corollary 4.8, for any e > 0, Ue(-) defined by (6.4) is the unique optimal control of 
Problem (SLQ)^ at (t,x) and 

(6.7) Ve{t,x) = Je(t,X]Ue{-)) ^V{t,x) + e'E |Me(s)|^ds. 

Also, we have 

(6.8) Vi;{t,x) ^ Je{t,x-,v*{-)) = V{t,x) + eE |z;*(s)pds. 

Combining (6.7)-(6.8), we have 

Ve{t,x) -V{t,x) 


(6.9) |it£(s)pds ^ 


^E^ |z;*(s)pds, Ve > 0. 


Thus, {Me(-)}e>o is bounded in the Hilbert space T] = L^{t, T ; R™) and hence admits a weakly convergent 
subsequence {ue^,{-)}k'^i. Let «*(•) be the weak limit of Since it(-) J{t,x;u{-)) is convex and 

continuous, it is hence sequentially weakly lower semi-continuous. Thus (noting (6.5)), 

V{t,x) J{t,x-,u*{-)) < lim J(t,x;uej,(-)) = V{t,x), 

k—^oo 

which implies that u*(-) is also an open-loop optimal control of Problem (SLQ) at {t,x). Now replacing v*{-) 
with u*{-) in (6.9), we have 

(6.10) E^ \us{s)\^ds ^E |M*(s)pds, Ver > 0. 

Also, by Lemma 6.3, part (i), 

(6.11) E f |u*(s)pds < lim E f |Me^(s)|^ds. 

J t k—J t 

Combining (6.10)-(6.11), we see that 


E [ |u*(s)pds = lim E f \u^^{s)\'^ds. 

Jt Jt 


Then it follows from Lemma 6.3, part (ii), that {ue^,{-)}k^i converges to u*{-) strongly. 
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(iii) => (ii) is obvious. 

(ii) ^ (i). Let {Mej, be a weakly convergent subsequence of {u£(-)}e>o with weak limit u*{-). Then 

is bounded in hl[t,T] = For any u{-) G U[t,T], we have 


( 6 . 12 ) 


J(t,a;;u(-))+efcE / \u{s)fds ^ Ve^{t,x) = J{t,x;Uei^{-)) + SkE / \ue,,{s)fds. 


Note that u(-) i—>■ J{t,x]u{-)) is sequentially weakly lower semi-continuous. Letting fc —>■ oo in (6.12), we 
obtain 

J{t,x;u*{-)) ^ lim J(t,x;uEj-)) ^ J(t,a;;u(-)), \/u{-) eU[t,T]. 

k—¥C!0 

Hence, u*{-) is an open-loop optimal control of Problem (SLQ) at {t,x). □ 


From the proof of Theorem 6.2, we see that the open-loop solvability of Problem (SLQ) at {t,x) is also 
equivalent the L^-boundedness of {'UE(-)}e>o- In particular, the open-loop solvability of Problem (SLQ)° at 
{t,x) is equivalent the L^-boundedness of {0e(-)Xe(-)}£>o with Xe(-) being the solution of 


(6.13) 


dX^{s) = {A + Be^)X^ds + {C + De^)X^dW{s), s G [t,T], 
Xe(t) = X. 


Since the Lp(H; C([t, T]; lR"))-norm of X^(-) is dominated by the L^-norm of 0 e(-)! we conjecture that the 
L^-boundedness of {0e(-)}e>o will lead to the open-loop solvability of Problem (SLQ)° at (t,x). Actually, 
we have the following result. 

Proposition 6.4. Let (H1)--(H2) hold. Suppose u{-) i—J°(0,0;u(-)) is convex, and let {0e(-)}e>o be 
the sequence defined by (6.3). If 

(6.14) sup / |0e(s)pcis < oo, 

e>0 JO 

then the Riccati equation (4.7) admits a regular solution P{-) G C'([0, T]; S”). Consequently, Problem (SLQ)° 
is closed-loop solvable. 

Proof. For any x G M" and e > 0, let Xs{-) be the solution of 


(6.15) 

By Ito’s formula, we have 


dXe{s) = [A(s)-I-i?(s)0e(s)]X£(s)ds-I- [C(s)-I-Zl(s)0£(s)]Xe(s)(ilF(s), sG [0,T], 
X,(0) = X. 


E\XS)\^ = |xp + eJ' { |[C(s) + i^(s)0e(s)]X,(s)|" + 2([A(s) + B{s)Q,is)]X,is), X,{s))}ds 


fo 

rt n 




\C{s) + D{s)Qe{s)\ + 2\A{s) + B{s)ee{s)\ E\X,{s)\^ds, \ft G [0,T]. 


Thus, by Gronwall’s inequality. 


(6.16) 


E|X,(t)|2 < Ixp exp I J [|C(s) -f 77 (s)0e(s)| 2 + 2|A(s) -f S(s)0e(s)| 
^ Ixpexp |a: ^1 + y |0E(s)pds^|, VtG[0,T], 


ds 
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where K > 0 is some constant depending only on A{-), D{-). Hence, 

e [ |0,(s)X,(s)|2ds^ / |0e(s)pE|xas)pds 

/a 1 ' 7 \ -^0 -^0 

|0e(s)pdsj|^ |0e(s)pds, 

which, together with (6.14), implies the L^-boundedness of {0£(s)Xe(s)}e>o. Thus, by Theorem 6.2, Problem 
(SLQ)° is open-loop solvable at t = 0, and by Theorem 5.3, Problem (SLQ)° is finite. Now let P : [0, T] —>■ §" 
such that (5.2) holds. Then by Theorem 5.3, 

(6.18) R + D^PD^O, a.e. 

Let {0efc(-)} be a weakly convergent subsequence of {0e(-)} with weak limit 0(-)- Since 

i? -I- el -b PgD R + PD as e —>■ 0 
and {!?(•) + el + D{-)^Ps{-)D{-)}o<e^i is uniformly bounded, we have 

+ D^Pe^C + S= -{R + Ekl + D^Pe^D)Qek ^ “(^ + PD)Q weakly in . 

Also, note that 

+ D^P^^C + S ^ B^P + D^PC + S strongly in L^. 

Thus, 

-(P -b D^PD)Q = B^P + D^PC + S. 

This implies 

(6.19) P(pTp -b D^PC -b S') C n{R + D^PD) , a.e. 

Since 

(P + pTpP)i(pTp + pTpC-bS) = -{R + PD)\r P PD)Q, 
and (P + PD^\r + PD^ is an orthogonal projection, we have 

(6.20) (P + pTpP)i(pTp + pTpC-bS) e p2(o,T;K™x"), 



and 


0 = -{R + PD^{B^P + PC + S) + [/- (P + pTpP)i(P + pTpP)]n 
for some n(-) G P^(0,T;K™^"). Finally, letting k —>■ oo, we have 


rT ^ 


P{t) — Pski^) — P'+ / Psk^P^ PekPP' C"-b Q-b (Pefc P-b C P^^D + S )0ej^ 

K—^OO K—^OO L 


ds 


rT , 


= G 
= G 


PA + A^P + G^PG -b g -b {PB + C^PD + S^)0 


ds 


PA+A ' P+G ' PC+Q-{PB+C ' PP-bS ' )(P+pTpP)i(pTp+P ' PG-bS) 


ds. 


which, together with (6.18)-(6.20), implies that P(-) is a regular solution of (4.7). 
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7 An example 


In this section we re-exam Example 2.2 to illustrate some results we obtained. In this example, the stochastic 
LQ problem admits a continuous open-loop optimal control at all {t,x) € [0,T) x R", hence it is open-loop 
solvable, while the value function is not continuous in t; the corresponding Riccati equation has a unique 
solution P(-), which does not satisfy the range condition (4.8) and therefore is not regular. Therefore, the 
problem is not closed-loop solvable on any [t, T]. This example also tells us that the necessity part in Theorem 
4.2 does not hold. 

Recall the following Problem (SLQ)° with one-dimensional state equation; 

I fiX(s) = [ui(s)-I-U2(s)]ds-I- [mi(s) - M2(s)]dhT(s), se[t, 1 ], 

[ X{t) = X, 

and cost functional 


(7.2) J°(t,x;u(-)) =EX(1)^. 

In this example, u(-) = (ui(-), U 2 (-))^ and 


71 = 0, B=(l,l), C = 0, B = 

'o 0^ 
0 0 , 


G = l, Q = 0, S'=(0,0)^, i? = 
The corresponding Riccati equation reads 


(7.3) 


p=p2(i,i)( 51 (!) = ^ ^ 


-p p 


-1 1 / \ i 


p(i) = i. 

Obviously, (7.3) has a unique solution P{-) = 1, and 


Tl{B{s)^P{s) + D{s)^P{s)C{s) + S{s)) = P((l, 1)^) = {(a,a)T : a S R} , 

'A -1^ ' 

-1 1 , 


n{R{s) -I- d{sY p{s)D{s)) = n 


= {(a, —a)^ : a G ' 


Thus, the range condition (4.8) does not hold and hence P(-) is not regular. By Theorem 4.4, the problem 
is not closed-loop solvable on any [t, 1]. 

Now for any e > 0, consider state equation (7.1) and the cost functional 


(7.4) J°(t,a:;u(-)) = E 
The Riccati equation for the above problem reads 

(7.5) 


X{l)'^ + eJ^ \u{s)\'^ds 



-p: 
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whose solution is given by 


(7.6) 

Letting e —>■ 0, we have 

(7.7) 


Peit) = 


e + 2-2r 


[ 0 , 1 ]. 


Po{t) = limPe(t) = 
£—>•0 


Thus, by Theorem 5.3, the original Problem (SLQ)^ is finite with value function 
(7.8) y°(t,x)=0, 0 < t < 1; 17°(1,x) = Vx G R. 

Next, set 

0 ^ 4 -{R + eI + D^P,D)-\B'^P, + D^PeC + S) = . 

Then the solution of 

dX^{s) = [A{s) + B{s)Qeis)]Xs{s)ds + [C{s) + D{s)Q^{s)]Xs{s)dW{s) 


0, 0 < t < 1, 

1, t = l. 


(7.9) 


2P 

= - -^Xs{s)ds, sG[t,T], 


Xe{t) = X 


V(s) = xexp 1“ ^ 


is given by 

(7.10) 
and hence 

(7.11) Ue(s) = 0e(s)X£(s) = - 
Note that for t G [0,1), 

Me(-) —>■ — 


2Peir) \ e + 2-2s 


dr > = 


£ + 2-2t 


e + 2-2p£ + 2-2t ’ 


X, t ^ s ^ 1, 


t ^ s ^ 1. 


X X 


2-2P 2-2t 


Thus, by Theorem 6.2, the original Problem (SLQ)° is open-loop solvable at any (t, x) G [0,T) x K with an 
open-loop optimal control 


in L as e —>■ 0. 


(7.12) 4_,)(s) = - (^ 

which is continuous in s G [t, 1]. 


X X 


2-2P 2-2t 


T 


t ^ S < 1, 
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